#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/title_happy.jpg"
Image(filename=image_path, width=600, height=600)
Happiness, a profound and elusive emotion, is the warm embrace of the soul when life's melodies harmonize with our inner symphony. It's the radiant sunbeam that pierces through the darkest of clouds, illuminating our path with hope and joy. Happiness is not a fleeting moment but a state of being, a tapestry woven from the threads of gratitude, contentment, and love.
It resides in the simple pleasures of life, like the laughter of children, the scent of blooming flowers, or the gentle touch of a loved one's hand. It's the satisfaction derived from pursuing our passions, the exhilaration of reaching new heights, and the serenity found in quiet moments of reflection.
Happiness is a choice, an art of embracing life's imperfections with a heart full of acceptance. It's the empathy we extend to others, the acts of kindness that create ripples of joy, and the connections we forge with fellow travelers on this journey.
In the tapestry of life, happiness is the vibrant thread that weaves through our experiences, making each moment precious. It's the compass that guides us towards a life well-lived, reminding us that true happiness is not found in possessions but in the richness of our human connections and the depths of our souls.
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
!pip install seaborn
Requirement already satisfied: seaborn in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (0.12.2) Requirement already satisfied: numpy!=1.24.0,>=1.17 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from seaborn) (1.24.3) Requirement already satisfied: pandas>=0.25 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from seaborn) (1.5.3) Requirement already satisfied: matplotlib!=3.6.1,>=3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from seaborn) (3.8.2) Requirement already satisfied: contourpy>=1.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (1.0.5) Requirement already satisfied: cycler>=0.10 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (0.11.0) Requirement already satisfied: fonttools>=4.22.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (4.25.0) Requirement already satisfied: kiwisolver>=1.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (1.4.4) Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (23.0) Requirement already satisfied: pillow>=8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (9.4.0) Requirement already satisfied: pyparsing>=2.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (3.0.9) Requirement already satisfied: python-dateutil>=2.7 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from pandas>=0.25->seaborn) (2022.7) Requirement already satisfied: six>=1.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib!=3.6.1,>=3.1->seaborn) (1.16.0)
cleaned_hm_csv_path = "/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv"
demographic_csv_path = "/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/demographic.csv"
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/happy_db.jpg"
Image(filename=image_path, width=600, height=600)
pip install wordcloud
Collecting wordcloud Obtaining dependency information for wordcloud from https://files.pythonhosted.org/packages/ac/66/937d1d73389c0b501c928c4d8513653063d2b40272dff70d0e283d8b9144/wordcloud-1.9.3-cp311-cp311-macosx_11_0_arm64.whl.metadata Downloading wordcloud-1.9.3-cp311-cp311-macosx_11_0_arm64.whl.metadata (3.4 kB) Requirement already satisfied: numpy>=1.6.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from wordcloud) (1.24.3) Requirement already satisfied: pillow in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from wordcloud) (9.4.0) Requirement already satisfied: matplotlib in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from wordcloud) (3.8.2) Requirement already satisfied: contourpy>=1.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (1.0.5) Requirement already satisfied: cycler>=0.10 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (0.11.0) Requirement already satisfied: fonttools>=4.22.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (4.25.0) Requirement already satisfied: kiwisolver>=1.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (1.4.4) Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (23.0) Requirement already satisfied: pyparsing>=2.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (3.0.9) Requirement already satisfied: python-dateutil>=2.7 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (2.8.2) Requirement already satisfied: six>=1.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib->wordcloud) (1.16.0) Downloading wordcloud-1.9.3-cp311-cp311-macosx_11_0_arm64.whl (168 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.3/168.3 kB 966.5 kB/s eta 0:00:00 0:00:01m Installing collected packages: wordcloud Successfully installed wordcloud-1.9.3 Note: you may need to restart the kernel to use updated packages.
import pandas as pd
from wordcloud import WordCloud
import matplotlib.pyplot as plt
# Read the CSV file
df = pd.read_csv('/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv')
# Concatenate text data from the 'cleaned_hm' column into a long string
text = ' '.join(df['cleaned_hm'].dropna())
# Generate a word cloud
wordcloud = WordCloud(width=800, height=400).generate(text)
# Display the word cloud image
plt.figure(figsize=(10, 7))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off') # Hide the axis
plt.show()
pip install spacy
Collecting spacy
Obtaining dependency information for spacy from https://files.pythonhosted.org/packages/ca/f3/609bb7512cad1f02af13daa23aa433b931da34c502211f29fd47dceff624/spacy-3.7.2-cp311-cp311-macosx_11_0_arm64.whl.metadata
Downloading spacy-3.7.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (25 kB)
Collecting spacy-legacy<3.1.0,>=3.0.11 (from spacy)
Downloading spacy_legacy-3.0.12-py2.py3-none-any.whl (29 kB)
Collecting spacy-loggers<2.0.0,>=1.0.0 (from spacy)
Obtaining dependency information for spacy-loggers<2.0.0,>=1.0.0 from https://files.pythonhosted.org/packages/33/78/d1a1a026ef3af911159398c939b1509d5c36fe524c7b644f34a5146c4e16/spacy_loggers-1.0.5-py3-none-any.whl.metadata
Downloading spacy_loggers-1.0.5-py3-none-any.whl.metadata (23 kB)
Collecting murmurhash<1.1.0,>=0.28.0 (from spacy)
Obtaining dependency information for murmurhash<1.1.0,>=0.28.0 from https://files.pythonhosted.org/packages/7a/05/4a3b5c3043c6d84c00bf0f574d326660702b1c10174fe6b44cef3c3dff08/murmurhash-1.0.10-cp311-cp311-macosx_11_0_arm64.whl.metadata
Downloading murmurhash-1.0.10-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.0 kB)
Collecting cymem<2.1.0,>=2.0.2 (from spacy)
Obtaining dependency information for cymem<2.1.0,>=2.0.2 from https://files.pythonhosted.org/packages/d7/f6/67babf1439cdd6d46e4e805616bee84981305c80e562320c293712f54034/cymem-2.0.8-cp311-cp311-macosx_11_0_arm64.whl.metadata
Downloading cymem-2.0.8-cp311-cp311-macosx_11_0_arm64.whl.metadata (8.4 kB)
Collecting preshed<3.1.0,>=3.0.2 (from spacy)
Obtaining dependency information for preshed<3.1.0,>=3.0.2 from https://files.pythonhosted.org/packages/a8/b3/1a73ba16bab53043fd19dd0a7838ae05c705dccb329404dd4ad5925767f1/preshed-3.0.9-cp311-cp311-macosx_11_0_arm64.whl.metadata
Downloading preshed-3.0.9-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.2 kB)
Collecting thinc<8.3.0,>=8.1.8 (from spacy)
Obtaining dependency information for thinc<8.3.0,>=8.1.8 from https://files.pythonhosted.org/packages/c8/43/0ee5796e9bf91eb8e1e032b4ef407e2fbec7cb2e3da9b13f44a2d2f82607/thinc-8.2.2-cp311-cp311-macosx_11_0_arm64.whl.metadata
Downloading thinc-8.2.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (15 kB)
Collecting wasabi<1.2.0,>=0.9.1 (from spacy)
Obtaining dependency information for wasabi<1.2.0,>=0.9.1 from https://files.pythonhosted.org/packages/8f/69/26cbf0bad11703241cb84d5324d868097f7a8faf2f1888354dac8883f3fc/wasabi-1.1.2-py3-none-any.whl.metadata
Downloading wasabi-1.1.2-py3-none-any.whl.metadata (28 kB)
Collecting srsly<3.0.0,>=2.4.3 (from spacy)
Obtaining dependency information for srsly<3.0.0,>=2.4.3 from https://files.pythonhosted.org/packages/1b/d7/0800af1a75008b3a6a6a24f3efd165f2d2208076e9b8a4b11b66f16217f3/srsly-2.4.8-cp311-cp311-macosx_11_0_arm64.whl.metadata
Downloading srsly-2.4.8-cp311-cp311-macosx_11_0_arm64.whl.metadata (20 kB)
Collecting catalogue<2.1.0,>=2.0.6 (from spacy)
Obtaining dependency information for catalogue<2.1.0,>=2.0.6 from https://files.pythonhosted.org/packages/9e/96/d32b941a501ab566a16358d68b6eb4e4acc373fab3c3c4d7d9e649f7b4bb/catalogue-2.0.10-py3-none-any.whl.metadata
Downloading catalogue-2.0.10-py3-none-any.whl.metadata (14 kB)
Collecting weasel<0.4.0,>=0.1.0 (from spacy)
Obtaining dependency information for weasel<0.4.0,>=0.1.0 from https://files.pythonhosted.org/packages/d5/e5/b63b8e255d89ba4155972990d42523251d4d1368c4906c646597f63870e2/weasel-0.3.4-py3-none-any.whl.metadata
Downloading weasel-0.3.4-py3-none-any.whl.metadata (4.7 kB)
Collecting typer<0.10.0,>=0.3.0 (from spacy)
Downloading typer-0.9.0-py3-none-any.whl (45 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.9/45.9 kB 2.6 MB/s eta 0:00:00
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (5.2.1)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (4.66.1)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.31.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.10.12)
Requirement already satisfied: jinja2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.1.2)
Requirement already satisfied: setuptools in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (68.0.0)
Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (23.0)
Collecting langcodes<4.0.0,>=3.2.0 (from spacy)
Downloading langcodes-3.3.0-py3-none-any.whl (181 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.6/181.6 kB 3.3 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: numpy>=1.19.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.24.3)
Requirement already satisfied: typing-extensions>=4.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (4.7.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2023.7.22)
Collecting blis<0.8.0,>=0.7.8 (from thinc<8.3.0,>=8.1.8->spacy)
Obtaining dependency information for blis<0.8.0,>=0.7.8 from https://files.pythonhosted.org/packages/a8/73/0a9d4e7f6e78ef270e3a4532b17e060a02087590cf615ba9943fd1a283e9/blis-0.7.11-cp311-cp311-macosx_11_0_arm64.whl.metadata
Downloading blis-0.7.11-cp311-cp311-macosx_11_0_arm64.whl.metadata (7.4 kB)
Collecting confection<1.0.0,>=0.0.1 (from thinc<8.3.0,>=8.1.8->spacy)
Obtaining dependency information for confection<1.0.0,>=0.0.1 from https://files.pythonhosted.org/packages/39/78/f9d18da7b979a2e6007bfcea2f3c8cc02ed210538ae1ce7e69092aed7b18/confection-0.1.4-py3-none-any.whl.metadata
Downloading confection-0.1.4-py3-none-any.whl.metadata (19 kB)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from typer<0.10.0,>=0.3.0->spacy) (8.0.4)
Collecting cloudpathlib<0.17.0,>=0.7.0 (from weasel<0.4.0,>=0.1.0->spacy)
Obtaining dependency information for cloudpathlib<0.17.0,>=0.7.0 from https://files.pythonhosted.org/packages/0f/6e/45b57a7d4573d85d0b0a39d99673dc1f5eea9d92a1a4603b35e968fbf89a/cloudpathlib-0.16.0-py3-none-any.whl.metadata
Downloading cloudpathlib-0.16.0-py3-none-any.whl.metadata (14 kB)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from jinja2->spacy) (2.1.1)
Downloading spacy-3.7.2-cp311-cp311-macosx_11_0_arm64.whl (6.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 27.1 MB/s eta 0:00:0000:0100:01
Downloading catalogue-2.0.10-py3-none-any.whl (17 kB)
Downloading cymem-2.0.8-cp311-cp311-macosx_11_0_arm64.whl (41 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.2/41.2 kB 8.3 MB/s eta 0:00:00
Downloading murmurhash-1.0.10-cp311-cp311-macosx_11_0_arm64.whl (26 kB)
Downloading preshed-3.0.9-cp311-cp311-macosx_11_0_arm64.whl (128 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.8/128.8 kB 28.6 MB/s eta 0:00:00
Downloading spacy_loggers-1.0.5-py3-none-any.whl (22 kB)
Downloading srsly-2.4.8-cp311-cp311-macosx_11_0_arm64.whl (488 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 488.4/488.4 kB 39.8 MB/s eta 0:00:00
Downloading thinc-8.2.2-cp311-cp311-macosx_11_0_arm64.whl (778 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 778.9/778.9 kB 36.2 MB/s eta 0:00:00
Downloading wasabi-1.1.2-py3-none-any.whl (27 kB)
Downloading weasel-0.3.4-py3-none-any.whl (50 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.1/50.1 kB 7.3 MB/s eta 0:00:00
Downloading blis-0.7.11-cp311-cp311-macosx_11_0_arm64.whl (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 44.6 MB/s eta 0:00:00
Downloading cloudpathlib-0.16.0-py3-none-any.whl (45 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.0/45.0 kB 7.4 MB/s eta 0:00:00
Downloading confection-0.1.4-py3-none-any.whl (35 kB)
Installing collected packages: cymem, wasabi, typer, spacy-loggers, spacy-legacy, murmurhash, langcodes, cloudpathlib, catalogue, blis, srsly, preshed, confection, weasel, thinc, spacy
Successfully installed blis-0.7.11 catalogue-2.0.10 cloudpathlib-0.16.0 confection-0.1.4 cymem-2.0.8 langcodes-3.3.0 murmurhash-1.0.10 preshed-3.0.9 spacy-3.7.2 spacy-legacy-3.0.12 spacy-loggers-1.0.5 srsly-2.4.8 thinc-8.2.2 typer-0.9.0 wasabi-1.1.2 weasel-0.3.4
Note: you may need to restart the kernel to use updated packages.
!pip install spacy wordcloud matplotlib
!python -m spacy download en_core_web_sm
Requirement already satisfied: spacy in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (3.7.2)
Requirement already satisfied: wordcloud in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (1.9.3)
Requirement already satisfied: matplotlib in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (3.8.2)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.1.8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (8.2.2)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (0.9.0)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (5.2.1)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (4.66.1)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.31.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.10.12)
Requirement already satisfied: jinja2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.1.2)
Requirement already satisfied: setuptools in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (68.0.0)
Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (23.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.3.0)
Requirement already satisfied: numpy>=1.19.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.24.3)
Requirement already satisfied: pillow in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from wordcloud) (9.4.0)
Requirement already satisfied: contourpy>=1.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (1.0.5)
Requirement already satisfied: cycler>=0.10 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (4.25.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (1.4.4)
Requirement already satisfied: pyparsing>=2.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (2.8.2)
Requirement already satisfied: typing-extensions>=4.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (4.7.1)
Requirement already satisfied: six>=1.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2023.7.22)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy) (0.1.4)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from typer<0.10.0,>=0.3.0->spacy) (8.0.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from weasel<0.4.0,>=0.1.0->spacy) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from jinja2->spacy) (2.1.1)
Collecting en-core-web-sm==3.7.1
Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl (12.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 46.6 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: spacy<3.8.0,>=3.7.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from en-core-web-sm==3.7.1) (3.7.2)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.1.8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (8.2.2)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.9.0)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (5.2.1)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (4.66.1)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.31.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.10.12)
Requirement already satisfied: jinja2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.1.2)
Requirement already satisfied: setuptools in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (68.0.0)
Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (23.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.3.0)
Requirement already satisfied: numpy>=1.19.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.24.3)
Requirement already satisfied: typing-extensions>=4.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (4.7.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2023.7.22)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.1.4)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from typer<0.10.0,>=0.3.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (8.0.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from weasel<0.4.0,>=0.1.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from jinja2->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.1.1)
Installing collected packages: en-core-web-sm
Successfully installed en-core-web-sm-3.7.1
✔ Download and installation successful
You can now load the package via spacy.load('en_core_web_sm')
import spacy
# Load the spaCy model
nlp = spacy.load("en_core_web_sm")
# Read the data
df = pd.read_csv('/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv')
# Concatenate all the text data
text = ' '.join(df['cleaned_hm'].dropna())
# Define a function to process large text
def process_large_text(text, chunk_size=1000000):
"""
Split a large text into smaller chunks and process each chunk using spaCy.
"""
nlp.max_length = chunk_size
doc_chunks = []
for i in range(0, len(text), chunk_size):
doc_chunk = nlp(text[i:i+chunk_size])
doc_chunks.append(doc_chunk)
return doc_chunks
# Call the function to process the text
doc_chunks = process_large_text(text)
# Now I can perform further processing on each document chunk, such as extracting information, generating word clouds, etc.
from wordcloud import WordCloud
import matplotlib.pyplot as plt
# Merge all the words from the document chunks
all_words = []
for doc_chunk in doc_chunks:
all_words.extend([token.text for token in doc_chunk])
# Concatenate the processed words into a long string
cleaned_text = ' '.join(all_words)
# Generate a word cloud using WordCloud, you can specify additional parameters like stopwords
wordcloud = WordCloud(
width=800,
height=400,
background_color='white'
).generate(cleaned_text)
# Display the generated word cloud
plt.figure(figsize=(15, 7))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()
from nltk.sentiment.vader import SentimentIntensityAnalyzer
from wordcloud import WordCloud
import matplotlib.pyplot as plt
# Initialize the sentiment analyzer
sia = SentimentIntensityAnalyzer()
# Read the data
df = pd.read_csv('/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv')
# Concatenate all the text data
text = ' '.join(df['cleaned_hm'].dropna())
# Perform sentiment analysis on the text and accumulate sentiment scores
word_sentiments = {} # Dictionary for word sentiments
for word in text.split():
sentiment = sia.polarity_scores(word)['compound']
word_sentiments[word] = word_sentiments.get(word, 0) + sentiment
# Generate a word cloud based on accumulated sentiment scores for words
wordcloud = WordCloud(
width=800,
height=400,
background_color='white'
).generate_from_frequencies(word_sentiments)
# Display the generated word cloud
plt.figure(figsize=(15, 7))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()
pip install textblob
Collecting textblob
Downloading textblob-0.17.1-py2.py3-none-any.whl (636 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 636.8/636.8 kB 2.6 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: nltk>=3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from textblob) (3.8.1)
Requirement already satisfied: click in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from nltk>=3.1->textblob) (8.0.4)
Requirement already satisfied: joblib in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from nltk>=3.1->textblob) (1.2.0)
Requirement already satisfied: regex>=2021.8.3 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from nltk>=3.1->textblob) (2022.7.9)
Requirement already satisfied: tqdm in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from nltk>=3.1->textblob) (4.66.1)
Installing collected packages: textblob
Successfully installed textblob-0.17.1
Note: you may need to restart the kernel to use updated packages.
from textblob import TextBlob
from wordcloud import WordCloud
import matplotlib.pyplot as plt
import nltk
from nltk.corpus import stopwords
# Download NLTK stopwords
nltk.download('stopwords')
# Read the data
df = pd.read_csv('/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv')
# Concatenate all the text data
text = ' '.join(df['cleaned_hm'].dropna())
# Perform sentiment analysis using TextBlob
blob = TextBlob(text)
# Get the sentiment score of the text
sentiment_score = blob.sentiment.polarity
# Tokenize the text
words = blob.words
# Remove stopwords
stop_words = set(stopwords.words('english'))
filtered_words = [word for word in words if word.lower() not in stop_words]
# Create a dictionary of word sentiments
word_sentiments = {}
for word in filtered_words:
word_sentiments[word] = blob.word_counts[word] * sentiment_score
# Generate a word cloud based on accumulated sentiment scores for words
wordcloud = WordCloud(
width=800,
height=400,
background_color='white'
).generate_from_frequencies(word_sentiments)
# Display the generated word cloud
plt.figure(figsize=(15, 7))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()
[nltk_data] Downloading package stopwords to [nltk_data] /Users/chenjianfeng/nltk_data... [nltk_data] Package stopwords is already up-to-date!
📒 In the HappyDB dataset, regardless of whether stop words are removed, we can distinctly see a few high-frequency words in the cumulative sentiment scores of the language model: work, friend, family, time, birthday, school, and more. This demonstrates that moments of happiness are ever-present in our lives. My parents often tell me that marriage is a significant lifelong investment that greatly influences one's happiness in the latter part of life, and I'm very much looking forward to finding a partner with whom I can grow together. Hence, my theme today revolves around the relationship between marriage and happiness. Follow in my footsteps! 👣
Here's a more idiomatic English version:
Within the HappyDB dataset, whether or not we filter out stop words, the sentiment scores from the language model clearly highlight several prevalent words: work, friend, family, time, birthday, school, among others. This underscores that moments of happiness are intertwined with our daily existence. My parents often emphasize that marriage is a profound lifelong commitment that significantly shapes our happiness in life's second half, and I eagerly anticipate finding a companion to grow alongside. Therefore, today's topic is the intricate connection between marriage and happiness. Let's embark on this journey together! 👣
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/marital_happy.jpg"
Image(filename=image_path, width=600, height=600)
read clean_hm.csv and degographic.csv file
clean_hm_df = pd.read_csv(cleaned_hm_csv_path)
clean_hm_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 27673 | 2053 | 24h | I went on a successful date with someone I fel... | I went on a successful date with someone I fel... | True | 1 | NaN | affection |
| 1 | 27674 | 2 | 24h | I was happy when my son got 90% marks in his e... | I was happy when my son got 90% marks in his e... | True | 1 | NaN | affection |
| 2 | 27675 | 1936 | 24h | I went to the gym this morning and did yoga. | I went to the gym this morning and did yoga. | True | 1 | NaN | exercise |
| 3 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding |
| 4 | 27677 | 6227 | 24h | I went with grandchildren to butterfly display... | I went with grandchildren to butterfly display... | True | 1 | NaN | affection |
demographic_df = pd.read_csv(demographic_csv_path)
demographic_df.head()
| wid | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y |
| 1 | 2 | 29.0 | IND | m | married | y |
| 2 | 3 | 25 | IND | m | single | n |
| 3 | 4 | 32 | USA | m | married | y |
| 4 | 5 | 29 | USA | m | married | y |
clean_hm_df.dropna(subset=['ground_truth_category'],inplace=True)
merge_df = demographic_df.merge(clean_hm_df,on='wid')
merge_df.head()
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y | 28054 | 24h | My mother called out of the blue to tell me ho... | My mother called out of the blue to tell me ho... | True | 1 | affection | affection |
| 1 | 1 | 37.0 | USA | m | married | y | 57637 | 24h | I got to take the day off from my part time jo... | I got to take the day off from my part time jo... | True | 1 | affection | affection |
| 2 | 1 | 37.0 | USA | m | married | y | 64498 | 3m | my son had a great time on his 8th birthday. | my son had a great time on his 8th birthday. | True | 1 | affection | affection |
| 3 | 1 | 37.0 | USA | m | married | y | 80306 | 3m | I got to have a three day weekend which i spen... | I got to have a three day weekend which i spen... | True | 1 | affection | affection |
| 4 | 1 | 37.0 | USA | m | married | y | 88307 | 3m | My fiance proposed to me in front of my family... | My fiance proposed to me in front of my family... | True | 1 | affection | affection |
merge_df = merge_df[merge_df['country'] == 'USA']
merge_df['age'] = pd.to_numeric(merge_df['age'], errors='coerce')
merge_df['age'] = merge_df['age'].fillna(0).astype(int)
adult_df = merge_df[merge_df['age']>16]
adult_df.head()
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37 | USA | m | married | y | 28054 | 24h | My mother called out of the blue to tell me ho... | My mother called out of the blue to tell me ho... | True | 1 | affection | affection |
| 1 | 1 | 37 | USA | m | married | y | 57637 | 24h | I got to take the day off from my part time jo... | I got to take the day off from my part time jo... | True | 1 | affection | affection |
| 2 | 1 | 37 | USA | m | married | y | 64498 | 3m | my son had a great time on his 8th birthday. | my son had a great time on his 8th birthday. | True | 1 | affection | affection |
| 3 | 1 | 37 | USA | m | married | y | 80306 | 3m | I got to have a three day weekend which i spen... | I got to have a three day weekend which i spen... | True | 1 | affection | affection |
| 4 | 1 | 37 | USA | m | married | y | 88307 | 3m | My fiance proposed to me in front of my family... | My fiance proposed to me in front of my family... | True | 1 | affection | affection |
adult_wid_unique_df = adult_df.drop_duplicates(subset=['wid'])
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/single_happy.jpg"
Image(filename=image_path, width=600, height=600)
marital_series = adult_wid_unique_df['marital'].value_counts()
sns.set_theme(style="whitegrid", palette="pastel")
barplot = sns.barplot(x=marital_series.index,y=marital_series.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
📒 The bar chart illustrates the number of individuals who experience happiness in various marital statuses. The data reveal that single individuals in the United States report a strong sense of happiness, with a proportion significantly higher than that of other groups. It is evident that the single population in America enjoys a level of happiness and joy that surpasses expectations.
Here's a more idiomatic English rewrite:
The bar graph presents the count of people experiencing joy across different marital statuses. It's clear from the data that those who are single in the U.S. have a notably robust sense of happiness, standing out with a significantly higher ratio than other demographics. This discovery highlights that the lives of American singles are filled with more happiness and joy than one might presume.
bins = [16, 20, 25, 30,35,40,45,50,55,60]
labels = ['16-20', '21-25', '26-30','31-35', '36-40', '41-45','46-50', '51-55', '56-60']
adult_wid_unique_df['range'] = pd.cut(adult_wid_unique_df['age'], bins=bins, labels=labels)
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_68554/3292791220.py:3: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy adult_wid_unique_df['range'] = pd.cut(adult_wid_unique_df['age'], bins=bins, labels=labels)
range_count_series = adult_wid_unique_df['range'].value_counts()
barplot = sns.barplot(x=range_count_series.index,y=range_count_series.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
The chart above segments data across age groups in increments of five years, from 16 to 60, creating nine distinct age bands. This stratification allows us to discern the age distribution of the survey respondents, revealing that a significant majority are concentrated in the 21 to 35 age range. This distribution could be indicative of the demographic makeup of the broader society.
Brainstorimg 🌟🌟🌟 : Supplement: You can dig deeper into this area, analyze the distribution of different proportions of men and women, and obtain the number of happy sentences for each age group to investigate the differences between men and women's happiness.
married_range_count = adult_wid_unique_df[adult_wid_unique_df['marital']=='married']['range'].value_counts()
married_range_count_dict = married_range_count.to_dict()
married_range_count_dict
{'31-35': 456,
'26-30': 435,
'36-40': 309,
'41-45': 172,
'21-25': 132,
'51-55': 96,
'46-50': 93,
'56-60': 62,
'16-20': 5}
married_range_count_updated_dict = {}
def update_list_to_cumulative_sum(lst):
for i in range(1, len(lst)):
lst[i] += lst[i - 1]
return lst
updated_list = update_list_to_cumulative_sum(list(married_range_count_dict.values()))
for idx,key in enumerate(married_range_count_dict.keys()):
married_range_count_updated_dict[key] = updated_list[idx]
married_range_count_series = pd.Series(married_range_count_updated_dict)
married_range_count_updated_dict
{'31-35': 456,
'26-30': 891,
'36-40': 1200,
'41-45': 1372,
'21-25': 1504,
'51-55': 1600,
'46-50': 1693,
'56-60': 1755,
'16-20': 1760}
g = sns.relplot(x=married_range_count_series.index,y=married_range_count_series.values,kind='scatter')
ax = g.axes[0,0]
ax.plot(married_range_count_series.index, married_range_count_series.values, color='gray', linestyle='--')
barplot = sns.barplot(x=married_range_count.index,y=married_range_count.values,ax=ax)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
📒From the perspective of the distribution of single people in various age ranges, first let's look at the histogram below, and combined with the distribution chart of single age ranges below, it is not difficult to see that the proportion of people aged 31-40 is larger than that of single people, which may be reflected from one side, and then After enjoying the happiness of single aristocrats, people in their 30s gradually began to consider marriage and family. Therefore, in the 30-40 range of the chart, the proportion of people who enjoy the happiness brought by marriage is larger than that of those who enjoy the happiness of being single. The above line chart also reflects from the side that people gradually consider marriage after the age of 20.
🌟🌟🌟Note: I just thought about this part and it still needs to be discussed. Later, I will further analyze the base ratio. It is okay not to explain the line chart. For the time being, it does not necessarily represent the growth trend of marriage.
sinle_range_count = adult_wid_unique_df[adult_wid_unique_df['marital']=='single']['range'].value_counts()
barplot = sns.barplot(x=sinle_range_count.index,y=sinle_range_count.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
📒Judging from the distribution of single people in various age ranges, it may reflect that in the 21-30 age range, young people are more enjoying the happiness brought by being single, and have not yet considered getting married, forming a family, or having children. Parenting.
adult_df.head()
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37 | USA | m | married | y | 28054 | 24h | My mother called out of the blue to tell me ho... | My mother called out of the blue to tell me ho... | True | 1 | affection | affection |
| 1 | 1 | 37 | USA | m | married | y | 57637 | 24h | I got to take the day off from my part time jo... | I got to take the day off from my part time jo... | True | 1 | affection | affection |
| 2 | 1 | 37 | USA | m | married | y | 64498 | 3m | my son had a great time on his 8th birthday. | my son had a great time on his 8th birthday. | True | 1 | affection | affection |
| 3 | 1 | 37 | USA | m | married | y | 80306 | 3m | I got to have a three day weekend which i spen... | I got to have a three day weekend which i spen... | True | 1 | affection | affection |
| 4 | 1 | 37 | USA | m | married | y | 88307 | 3m | My fiance proposed to me in front of my family... | My fiance proposed to me in front of my family... | True | 1 | affection | affection |
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/career_happy.jpg"
Image(filename=image_path, width=600, height=600)
single_series = adult_df[adult_df['marital']=='single']['predicted_category'].value_counts()
barplot = sns.barplot(x=single_series.index,y=single_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=90)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
📒This is a distribution map of different happiness types of single people. Comparing the figure below, it is easy to see that single young people are better than married people in terms of exercise, leisure, enjoy_the_moment, bonding, etc. Single people are far better than married people of the same age. They are more keen on various social activities, such as dining out, exercising, taking elective art and music courses, participating in various public and charity activities, etc. In particular, the achievement item is higher than that of married people, which shows that they still focus on career and value career and academic achievements more.
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/parenthood_happy.jpg"
Image(filename=image_path, width=600, height=600)
married_series = adult_df[adult_df['marital']=='married']['predicted_category'].value_counts()
barplot = sns.barplot(x=married_series.index,y=married_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=90)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
📒This is a distribution map of different happiness types of single people. Comparing the figure below, it is easy to see that single young people are better than married people in terms of exercise, leisure, enjoy_the_moment, bonding, etc. Single people are far better than married people of the same age. They are more keen on various social activities, such as dining out, exercising, taking elective art and music courses, participating in various public and charity activities, etc. In particular, the achievement item is higher than that of married people, which shows that they still focus on career and value career and academic achievements more.
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/partner_happy.jpg"
Image(filename=image_path, width=600, height=600)
married_parenthood_series = adult_df[adult_df['marital']=='married']['parenthood'].value_counts()
barplot = sns.barplot(x=married_parenthood_series.index,y=married_parenthood_series.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
📒Among married people, comparing those with children and those without children, it is not difficult to see that the proportion of married people with children is larger. Maybe the source of happiness for married people is affection, and affection may be spending time with family members. Sharing happy times together may make people with children feel more happy. This may also be one of the reasons for forming a family and getting married.
df = pd.read_csv("sementic_cleaned_hm.csv")
df.head()
demographic_csv_path = "/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/demographic.csv"
demographic_df = pd.read_csv(demographic_csv_path)
demographic_df.head()
| wid | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y |
| 1 | 2 | 29.0 | IND | m | married | y |
| 2 | 3 | 25 | IND | m | single | n |
| 3 | 4 | 32 | USA | m | married | y |
| 4 | 5 | 29 | USA | m | married | y |
sns.set_theme(style="whitegrid", palette="pastel")
ax = sns.boxplot(x='predicted_category', y='positive', data=df)
ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
C:\Users\yhyxx\AppData\Local\Temp\ipykernel_6836\268421861.py:3: UserWarning: FixedFormatter should only be used together with FixedLocator ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
[Text(0, 0, 'affection'), Text(1, 0, 'exercise'), Text(2, 0, 'bonding'), Text(3, 0, 'leisure'), Text(4, 0, 'achievement'), Text(5, 0, 'enjoy_the_moment'), Text(6, 0, 'nature')]
df_ground_truth = df.dropna(subset=['ground_truth_category'])
ax = sns.boxplot(x='predicted_category', y='positive', data=df_ground_truth)
ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
C:\Users\yhyxx\AppData\Local\Temp\ipykernel_6836\2889382971.py:2: UserWarning: FixedFormatter should only be used together with FixedLocator ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
[Text(0, 0, 'bonding'), Text(1, 0, 'leisure'), Text(2, 0, 'affection'), Text(3, 0, 'enjoy_the_moment'), Text(4, 0, 'achievement'), Text(5, 0, 'nature'), Text(6, 0, 'exercise')]
ax = sns.boxplot(x='predicted_category', y='negative', data=df_ground_truth)
ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
C:\Users\yhyxx\AppData\Local\Temp\ipykernel_6836\1198910182.py:2: UserWarning: FixedFormatter should only be used together with FixedLocator ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
[Text(0, 0, 'bonding'), Text(1, 0, 'leisure'), Text(2, 0, 'affection'), Text(3, 0, 'enjoy_the_moment'), Text(4, 0, 'achievement'), Text(5, 0, 'nature'), Text(6, 0, 'exercise')]
merge_df = df_ground_truth.merge(demographic_df,on='wid')
merge_df.head()
| Unnamed: 0 | hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | positive | negative | neutral | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 3 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding | 0.954003 | 0.044106 | 0.001891 | 28 | DNK | f | married | n |
| 1 | 21745 | 49534 | 206 | 24h | I bought my wedding shoes! | I bought my wedding shoes! | True | 1 | achievement | achievement | 0.221079 | 0.271567 | 0.507355 | 28 | DNK | f | married | n |
| 2 | 24645 | 52451 | 206 | 24h | My new watch arrived in the mail! | My new watch arrived in the mail! | True | 1 | achievement | achievement | 0.146610 | 0.213970 | 0.639420 | 28 | DNK | f | married | n |
| 3 | 56163 | 84125 | 206 | 3m | We watched Rocky Horror Picture Show for the f... | We watched Rocky Horror Picture Show for the f... | True | 1 | leisure | leisure | 0.717492 | 0.189807 | 0.092702 | 28 | DNK | f | married | n |
| 4 | 57156 | 85125 | 206 | 3m | We played Mario Kart while drunk. | We played Mario Kart while drunk. | True | 1 | leisure | leisure | 0.649013 | 0.241162 | 0.109825 | 28 | DNK | f | married | n |
merge_df['age'] = pd.to_numeric(merge_df['age'], errors='coerce')
bins = [16, 20, 25, 30,35,40,45,50,55,60]
labels = ['16-20', '21-25', '26-30','31-35', '36-40', '41-45','46-50', '51-55', '56-60']
merge_df['range'] = pd.cut(merge_df['age'], bins=bins, labels=labels)
df_age_category_df = merge_df[['ground_truth_category','range','positive']]
df_age_category_df.head()
| ground_truth_category | range | positive | |
|---|---|---|---|
| 0 | bonding | 26-30 | 0.954003 |
| 1 | achievement | 26-30 | 0.221079 |
| 2 | achievement | 26-30 | 0.146610 |
| 3 | leisure | 26-30 | 0.717492 |
| 4 | leisure | 26-30 | 0.649013 |
grouped_category_age = df_age_category_df.groupby(['range', 'ground_truth_category']).mean()
pivot_table_category_age = grouped_category_age.pivot_table(index='ground_truth_category', columns='range', values='positive')
sns.heatmap(pivot_table_category_age, annot=True, cmap="YlGnBu")
<Axes: xlabel='range', ylabel='ground_truth_category'>
📒It is not difficult for us to find in the heat map after weighting the emotional scoring model, whether it is 16-20, 21-25, 26-30, 36-40, 41-45, 46-50, 51-55, 56-60 age range Bonding has the strongest correlation, but based on the statistics of the number of happy moments we obtained above, "the biggest source of happiness for married people is affections" and "the biggest source of happiness for single people is achievement". These results are contrary to each other, so we judge this The emotional scoring model may not be suitable for our data. We can try more emotional scoring models in the future, such as roberta, spaCy, etc.
📒 From here we can be more confident that the emotion scoring model we chose has increased the outliers of the data to a certain extent.
When using sentiment models to score data and then creating box plots for weighted statistics, the presence of a large number of outliers can occur for several reasons:
Skewed Data: The underlying data might be skewed. Sentiment scores often have a non-normal distribution, especially in datasets with diverse expressions of sentiment that can lead to a heavy concentration of scores at one end of the spectrum.
Sensitive Scoring Model: The sentiment model used for scoring might be very sensitive to certain keywords or phrases, causing a wider dispersion of sentiment scores that are not representative of the general sentiment.
Diverse Sentiment Expression: People express sentiments in very different ways. Some expressions may be very mild, while others are extremely positive or negative, which can lead to scores that are significantly higher or lower than the median.
Weighting Issues: If the statistical weighting is not applied correctly or the weights themselves are not representative of the population, it can distort the sentiment scores and lead to outliers.
Data Entry Errors: Sometimes, outliers are simply the result of data entry errors or anomalies in the data collection process.
Narrow Interquartile Range (IQR): A box plot displays outliers based on the IQR. If the IQR is narrow due to a large clustering of data within a specific sentiment score range, then even moderately deviant scores can appear as outliers.
To address this, you might consider the following approaches:
Data Cleaning: Ensure that the data is cleaned properly to remove any errors that might cause outliers.
Model Calibration: Adjust the sentiment model to be less sensitive to outliers or to better accommodate the range of sentiment expressions in your data.
Analysis of Outliers: Conduct a thorough analysis of the outliers to understand why they are occurring. This could involve a qualitative review of the high and low sentiment texts.
Robust Statistical Methods: Use statistical methods that are less affected by outliers, or adjust the definition of outliers in the context of your data.
import torch
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from tqdm.notebook import tqdm
import numpy as np
from scipy.spatial.distance import cdist
sementic_df = pd.read_csv("/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/whole/sementic_cleaned_hm.csv")
demographic_csv_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv"
demographic_df = pd.read_csv(demographic_csv_path)
demographic_df.head()
| wid | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y |
| 1 | 2 | 29.0 | IND | m | married | y |
| 2 | 3 | 25 | IND | m | single | n |
| 3 | 4 | 32 | USA | m | married | y |
| 4 | 5 | 29 | USA | m | married | y |
df_ground_truth = sementic_df.dropna(subset=['ground_truth_category'])
merge_df = df_ground_truth.merge(demographic_df,on='wid')
len(merge_df)
14125
merge_df.head()
| Unnamed: 0 | hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | positive | negative | neutral | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 3 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding | 0.954003 | 0.044106 | 0.001891 | 28 | DNK | f | married | n |
| 1 | 21745 | 49534 | 206 | 24h | I bought my wedding shoes! | I bought my wedding shoes! | True | 1 | achievement | achievement | 0.221079 | 0.271567 | 0.507355 | 28 | DNK | f | married | n |
| 2 | 24645 | 52451 | 206 | 24h | My new watch arrived in the mail! | My new watch arrived in the mail! | True | 1 | achievement | achievement | 0.146610 | 0.213970 | 0.639420 | 28 | DNK | f | married | n |
| 3 | 56163 | 84125 | 206 | 3m | We watched Rocky Horror Picture Show for the f... | We watched Rocky Horror Picture Show for the f... | True | 1 | leisure | leisure | 0.717492 | 0.189807 | 0.092702 | 28 | DNK | f | married | n |
| 4 | 57156 | 85125 | 206 | 3m | We played Mario Kart while drunk. | We played Mario Kart while drunk. | True | 1 | leisure | leisure | 0.649013 | 0.241162 | 0.109825 | 28 | DNK | f | married | n |
merge_df = merge_df[merge_df['country']=='USA']
merge_df.head()
| Unnamed: 0 | hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | positive | negative | neutral | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 16 | 32 | 27705 | 5732 | 24h | I picked my daughter up from the airport and w... | I picked my daughter up from the airport and w... | True | 1 | bonding | affection | 0.974089 | 0.016479 | 0.009433 | 64 | USA | f | divorced | y |
| 17 | 42 | 27715 | 2272 | 24h | when i received flowers from my best friend | when i received flowers from my best friend | True | 1 | bonding | bonding | 0.987854 | 0.000706 | 0.011440 | 27 | USA | m | single | n |
| 18 | 8526 | 36258 | 2272 | 24h | while meeting a friend i laughed so much that ... | while meeting a friend i laughed so much that ... | True | 1 | bonding | bonding | 0.970213 | 0.015644 | 0.014143 | 27 | USA | m | single | n |
| 19 | 14339 | 42099 | 2272 | 24h | i bought a new car for which i saved for a ver... | i bought a new car for which i saved for a ver... | True | 1 | achievement | achievement | 0.789608 | 0.117012 | 0.093379 | 27 | USA | m | single | n |
| 20 | 50 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 0.126672 | 0.128704 | 0.744623 | 61 | USA | f | married | y |
temp_df = merge_df[['wid','ground_truth_category','positive']]
temp_df.head()
| wid | ground_truth_category | positive | |
|---|---|---|---|
| 16 | 5732 | bonding | 0.974089 |
| 17 | 2272 | bonding | 0.987854 |
| 18 | 2272 | bonding | 0.970213 |
| 19 | 2272 | achievement | 0.789608 |
| 20 | 389 | leisure | 0.126672 |
temp_df['ground_truth_category'].unique()
array(['bonding', 'achievement', 'leisure', 'affection',
'enjoy_the_moment', 'exercise', 'nature'], dtype=object)
temp_df_grp = temp_df.groupby(['wid'])
temp_df_grp
<pandas.core.groupby.generic.DataFrameGroupBy object at 0x15c1c4950>
records = []
for name,group in tqdm(temp_df_grp):
rec = {
"wid":name
}
arr = []
for category in ['bonding', 'achievement', 'leisure', 'affection','enjoy_the_moment', 'exercise', 'nature']:
# print(len(group[group['ground_truth_category']==category]))
if (len(group[group['ground_truth_category']==category])> 0):
# print(f"mean of {category} = {group[group['ground_truth_category']==category]['positive'].mean()}")
arr.append(group[group['ground_truth_category']==category]['positive'].mean())
else:
arr.append(0)
rec['data'] = arr
records.append(rec)
0%| | 0/4644 [00:00<?, ?it/s]
/Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/tqdm/std.py:1182: FutureWarning: In a future version of pandas, a length 1 tuple will be returned when iterating over a groupby with a grouper equal to a list of length 1. Don't supply a list with a single grouper to avoid this warning. for obj in iterable:
workers_df = pd.DataFrame(records)
workers_df.head()
| wid | data | |
|---|---|---|
| 0 | 1 | [0, 0.137100713327527, 0, 0.4215105965733528, ... |
| 1 | 4 | [0, 0.29191372791926057, 0.30610373616218567, ... |
| 2 | 5 | [0, 0.9422732506479535, 0.9660078287124634, 0.... |
| 3 | 7 | [0, 0.9755759239196777, 0, 0.6728441839416822,... |
| 4 | 9 | [0, 0.682466983795166, 0.7587945759296417, 0.3... |
workers_df['data'].tolist()
[[0, 0.137100713327527, 0, 0.4215105965733528, 0, 0, 0], [0, 0.29191372791926057, 0.30610373616218567, 0.5476994067430496, 0.9841907620429992, 0, 0], [0, 0.9422732506479535, 0.9660078287124634, 0.7193172574043274, 0.863121509552002, 0, 0], [0, 0.9755759239196777, 0, 0.6728441839416822, 0, 0, 0], [0, 0.682466983795166, 0.7587945759296417, 0.3264326453208923, 0, 0, 0.3608618378639221], [0.999874711036682, 0.8684602975845337, 0, 0.8958892822265625, 0, 0.1307117491960525, 0], [0, 0.6169674638658762, 0.7633823752403259, 0.6480808854103088, 0.1526822745800018, 0.6617514193058014, 0.25640395283699036], [0.9975021481513976, 0.9728579223155975, 0, 0.9819894518171038, 0, 0, 0], [0, 0.6628201588988304, 0, 0.8284440040588379, 0.3824873268604278, 0, 0], [0.7689815759658813, 0.3922153562307358, 0.2177372723817825, 0.3639296293258667, 0.8849262595176697, 0, 0], [0.9059566855430603, 0.7955472022294998, 0, 0.840456485748291, 0.5624952912330627, 0, 0.0938845723867416], [0.8958899080753326, 0.26298290491104126, 0, 0.18592155724763865, 0, 0, 0], [0.9556574622790018, 0.6089979360500971, 0.799452930688858, 0, 0.6815130323171615, 0, 0], [0, 0.46578184288481006, 0, 0.6745045185089111, 0, 0, 0], [0.9483643770217896, 0.40721282901035416, 0.472448468208313, 0, 0.6506055692831675, 0, 0], [0.9301214416821798, 0.8769465088844299, 0.9812466502189636, 0.0685957819223404, 0, 0.3826212584972381, 0], [0.4926667511463165, 0.4713267336289088, 0, 0.5170791536569594, 0.296459436416626, 0, 0], [0, 0.5040227721134821, 0.2419122159481048, 0.8007865846157074, 0.6226471761862437, 0, 0], [0.8374781608581543, 0.2978727122147878, 0.5873757004737854, 0.2808998748660087, 0.8766465187072754, 0, 0], [0, 0.4748096466064453, 0.7896565794944763, 0, 0.964567005634308, 0, 0], [0.9215424458185831, 0.6832125782966614, 0, 0.5663713216781616, 0, 0, 0], [0, 0.6933804601430893, 0, 0.39410303533077234, 0.9034669548273087, 0, 0.8658231496810913], [0.9999866485595704, 0.5502831128736337, 0, 0, 0.5073545873165131, 0.185338944196701, 0], [0, 0.7544189356267452, 0.7007513493299484, 0.4470658302307129, 0.8755398094654083, 0, 0], [0.988240122795105, 0.415598937869072, 0, 0.9659907817840576, 0.9083369970321655, 0.948187291622162, 0], [0.9928910732269288, 0.754799447953701, 0, 0.5756111547350884, 0, 0, 0], [0.996465802192688, 0.8663649260997772, 0, 0.9230564832687378, 0.988989531993866, 0, 0], [0.239871472120285, 0.459239199757576, 0, 0.5011677853763103, 0, 0.1856067031621933, 0], [0.8944230079650879, 0.733821451663971, 0, 0, 0.731142520904541, 0, 0], [0.9703196287155152, 0, 0.625187873840332, 0.7831381956736246, 0.9231996238231659, 0.1033431366086006, 0], [0, 0, 0, 0.6917418837547302, 0.7850021762507302, 0.0383162312209606, 0], [0.79929518699646, 0.7108263969421387, 0.6523396223783493, 0.3196532428264618, 0.7945820093154907, 0, 0], [0.8192546740174294, 0.8645654320716858, 0.5079385042190552, 0.7089867293834686, 0, 0, 0], [0.6028576493263245, 0.46833112090826035, 0, 0.2387157082557678, 0, 0.2868005037307739, 0], [0.8783846696217855, 0.2565247118473053, 0, 0.591572953077654, 0, 0, 0], [0.9891067147254944, 0.9434030652046204, 0.8670592308044434, 0.7326608151197433, 0.0796752274036407, 0, 0], [0.828554759422938, 0.7491686791181564, 0.9996336698532104, 0.3036886490881443, 0.755229115486145, 0, 0.9890613555908204], [0, 0.29721872955560685, 0, 0.5482053607702255, 0, 0, 0], [0.9569055438041688, 0.7566386163234711, 0, 0.720783531665802, 0.8449609676996866, 0, 0], [0.9926135540008544, 0.3399884030222892, 0.1189094111323356, 0.541741356253624, 0.3728630542755127, 0, 0.7974621057510376], [0.9968536496162416, 0.4969925781091054, 0.6155764535069466, 0.1455091387033462, 0.1779616773128509, 0, 0], [0, 0.6371815800666809, 0.9668627977371216, 0.5996158868074417, 0, 0, 0], [0, 0.9189724624156952, 0, 0.3418990690261125, 0, 0, 0], [0.9984659850597382, 0.21195042133331293, 0.5038295984268188, 0, 0, 0, 0], [0, 0.6322824135422707, 0.87706059217453, 0.682238906621933, 0.961948424577713, 0.9209374189376832, 0], [0, 0.6114794611930847, 0.98650324344635, 0.4198198430240154, 0.4572100043296814, 0, 0], [0.8902253210544586, 0.34767340496182436, 0, 0.8208476305007935, 0, 0, 0], [0, 0.4214778078926934, 0.49575117230415344, 0, 0, 0, 0.7239409685134888], [0, 0.0988653376698494, 0.7525047461191813, 0.5598754733800888, 0.8871830105781555, 0, 0.4831818044185638], [0.4350074231624603, 0.7171584644487926, 0.022810960188508, 0, 0.7467517852783203, 0, 0.7174817472696304], [0, 0, 0.9738746881484984, 0, 0.9920459985733032, 0, 0], [0.6648801267147064, 0.6031370013952255, 0, 0.40263592700163525, 0.2723465263843536, 0, 0], [0, 0.11487837508320806, 0.7907939553260803, 0.17696680980069296, 0, 0, 0], [0, 0.5146242141723633, 0, 0.1753241841991742, 0.2968374788761139, 0, 0.6932640075683594], [0.650206983089447, 0.23416271805763242, 0.4285113364458084, 0.7685751914978027, 0.4929863587021827, 0, 0], [0.9458845257759094, 0.9476355910301208, 0, 0.8945919275283813, 0.6802728772163391, 0, 0], [0, 0.17768862595160798, 0, 0.22141575174672262, 0, 0, 0], [0, 0.45381397008895874, 0, 0.3482124879956245, 0.5118028372526169, 0, 0], [0, 0.5522683560848236, 0.7328639030456543, 0.300720501691103, 0.3928524851799011, 0, 0], [0.4916459918022156, 0.5599977572758993, 0.7413403987884521, 0.435212604701519, 0.8937118053436279, 0, 0], [0, 0.3788133561611175, 0.5468888580799102, 0.2893667891621589, 0.7007676362991333, 0.921634554862976, 0], [0.7398422956466675, 0, 0.5515993013978004, 0.3945907255013783, 0.6331590414047241, 0, 0], [0.9644231001536051, 0.47703437010447186, 0, 0.3521240353584289, 0, 0, 0], [0, 0.8495193421840668, 0, 0.3973444650570552, 0, 0, 0], [0, 0.46979445219039917, 0, 0.4651801586151123, 0.4979938517014186, 0, 0], [0.7197631001472473, 0.2750847302377224, 0.5360426306724548, 0.0950754359364509, 0, 0, 0.3229615315794945], [0, 0.7172285914421082, 0.1054267138242721, 0, 0, 0, 0.9700478315353394], [0, 0.6648417413234711, 0, 0.26936504803597927, 0, 0, 0], [0, 0.5346575975418091, 0.4748132824897766, 0.8278654038906097, 0.9479748606681824, 0, 0.9699013829231262], [0, 0.8047166168689728, 0, 0, 0.974994421005249, 0.8144822120666504, 0.9963191747665404], [0, 0.6246875077486038, 0, 0.19577843944231665, 0, 0, 0], [0.8359584957361221, 0.591045469045639, 0.22885722666978836, 0, 0, 0, 0.1401931345462799], [0.8999206125736237, 0.34735628465811413, 0, 0.45939776860177517, 0.9970507621765136, 0, 0], [0.9001186490058899, 0.2257867008447647, 0.0536536835134029, 0, 0, 0, 0], [0.9446530342102052, 0, 0, 0.7486021518707275, 0.4959306120872497, 0, 0.8014942407608032], [0.8392297029495239, 0.3910011649131775, 0.37254180759191513, 0.5677797198295593, 0, 0, 0], [0, 0.3474336490035057, 0, 0.7773531153798103, 0.5059447884559631, 0, 0], [0, 0, 0.3284018412232399, 0.25027199275791645, 0.2157990038394928, 0, 0], [0.6435366868972778, 0, 0, 0.47761650880177814, 0, 0, 0.8055709004402161], [0, 0.579346239566803, 0, 0.8424153923988342, 0, 0, 0], [0, 0.3060519084334373, 0.8512274026870728, 0, 0, 0, 0], [0.6645182967185974, 0.7834499080975851, 0, 0.6732126375039419, 0, 0, 0], [0, 0.3370041251182556, 0.7473838150501251, 0.1194042563438415, 0.2176694422960281, 0, 0.1381328850984573], [0.8452879985173544, 0.3196448404341936, 0.8344496488571167, 0.2587212920188904, 0, 0, 0], [0, 0.2357016056776046, 0.8540000518163046, 0.25071811676025385, 0.8572898507118225, 0, 0], [0.8726728558540344, 0.329301701237758, 0, 0, 0, 0, 0], [0.7184855937957764, 0.2704422175884247, 0, 0.3286152631044387, 0.5623554686705271, 0, 0], [0, 0.43621434768040973, 0.4896071553230285, 0.5636279359459876, 0.9276829361915588, 0, 0], [0.4683143546183903, 0.698274552822113, 0.8982224464416504, 0.5514293797314167, 0, 0.987236499786377, 0], [0.9644092321395874, 0.8495615273714066, 0, 0.8226788341999054, 0, 0, 0], [0.7226612567901611, 0.1379027366638183, 0, 0.94182687997818, 0, 0.3588936987022559, 0], [0.9578983783721924, 0.929506242275238, 0, 0.3568437462672591, 0, 0, 0], [0.8145684003829956, 0.3032463043928146, 0, 0.7205374588569006, 0.2029759734869003, 0.0958877354860305, 0], [0.8606164753437042, 0.3879231661558151, 0, 0.2935197651386261, 0.5523084998130798, 0, 0], [0, 0.8642290234565735, 0, 0.3759870618581772, 0.2835976481437683, 0, 0.545119047164917], [0, 0.4937963187694549, 0, 0.36191037297248835, 0, 0.2885719835758209, 0.3577573895454407], [0.4901515692472458, 0.7270233631134033, 0, 0.18692402401939032, 0.9420002102851868, 0, 0], [0.9879974126815796, 0, 0.9021123349666595, 0.747507855296135, 0.9284858902295431, 0.8776967525482178, 0.9994885921478271], [0, 0.4052640981972217, 0.9969916343688964, 0.6238415514429411, 0, 0, 0], [0.8169680833816528, 0.8370913167794546, 0, 0.768219788869222, 0, 0, 0.7710949778556824], [0.9829716086387634, 0.15124856680631632, 0, 0, 0, 0.0265612136572599, 0], [0.0693808421492576, 0.5253352671861649, 0, 0.3593113929033279, 0, 0, 0], [0.82593834400177, 0.3369923746213317, 0, 0, 0.991141676902771, 0, 0.34272746245066327], [0, 0.26349432642261184, 0, 0.1301777213811874, 0.5117710530757904, 0, 0], [0, 0.2685096887871623, 0, 0.0737054869532585, 0, 0, 0], [0.8726977904637655, 0.3881535828113556, 0, 0.5879924595355988, 0, 0, 0], [0.8911462426185608, 0.4691902697086334, 0.5983064949512482, 0.1079471632838249, 0, 0, 0], [0.6712708324193954, 0.6674621313810348, 0.5995742082595825, 0.7495078444480896, 0, 0, 0], [0, 0.5911137089133263, 0, 0.9157728850841522, 0.8018226027488708, 0, 0], [0, 0.37774621446927387, 0, 0.6061648726463318, 0, 0, 0], [0, 0.5572262480854988, 0.3542337417602539, 0.4738970827311277, 0, 0, 0], [0, 0.544276624917984, 0.0238804724067449, 0, 0, 0, 0], [0.9548856168985367, 0, 0, 0.7844707022110621, 0, 0.9943504929542542, 0], [0, 0.4298538684844971, 0.2555645555257797, 0, 0, 0, 0], [0, 0.5001437415679296, 0.2018689513206482, 0.9052593410015106, 0.711388885974884, 0, 0], [0.8481751084327698, 0.2106828242540359, 0.1391331851482391, 0.17104310914874074, 0.8073089718818665, 0, 0], [0.864166647195816, 0, 0.48734255135059357, 0, 0.6239410638809204, 0.5981636941432953, 0], [0.9122249633073807, 0.1571846157312393, 0.58230921626091, 0.3338982164859772, 0.7135291695594788, 0, 0.9963982105255128], [0, 0.54918172955513, 0, 0, 0.7919442057609558, 0, 0], [0.9926827748616537, 0.42959941178560257, 0.363324910402298, 0.647139290968577, 0.9718157052993774, 0, 0], [0.9142643213272095, 0.5984932680924734, 0.081913910806179, 0.2149767875671386, 0, 0.0529461875557899, 0], [0.5008736634626985, 0.12081726330022013, 0, 0, 0.3371348679065704, 0, 0], [0.7376567870378494, 0.2076014926036199, 0.8949919939041138, 0.15126771790285906, 0.6624586582183838, 0, 0], [0.9660285115242004, 0.7882741093635559, 0, 0.7800138294696808, 0.9869790077209472, 0, 0], [0.9972072243690492, 0.9734096527099608, 0, 0.8973030209541321, 0, 0, 0], [0, 0.39572038253148395, 0, 0.5058121494948864, 0.4436399936676025, 0, 0], [0.7357881516218185, 0, 0, 0, 0.8267797827720642, 0.2264408841729164, 0], [0, 0.4634135738015175, 0, 0.8460221886634827, 0.7346308728059133, 0, 0.9397295117378236], [0.9948845505714417, 0.700884997844696, 0, 0.5673792362213135, 0, 0, 0], [0, 0.10081899911165235, 0, 0, 0, 0, 0], [0, 0, 0.1861652582883834, 0.7361492365598679, 0.0097096590325236, 0, 0], [0, 0.528712160885334, 0.19506651908159256, 0.41507573425769806, 0, 0, 0], [0, 0.5574690699577332, 0.111922986805439, 0, 0, 0, 0], [0.8509494185447692, 0.5611602465311686, 0.9535676538944244, 0, 0.3331254422664642, 0, 0], [0.7822547256946564, 0.49072160199284554, 0, 0.4295258820056915, 0.9813901782035828, 0, 0.1755695044994354], [0.804538905620575, 0.3813246600329876, 0, 0, 0, 0, 0], [0, 0.22208910435438156, 0.6472277045249939, 0.6127094253897667, 0, 0, 0], [0, 0.7202769368886948, 0, 0.8603561595082283, 0.3729786276817322, 0, 0.4384655952453613], [0, 0.28686569631099695, 0, 0.393586240708828, 0, 0, 0], [0, 0.9929047226905824, 0, 0.5404271334409714, 0.8398195505142212, 0, 0], [0.9185660779476166, 0.21479429677128792, 0.6849555373191833, 0, 0.1801740229129791, 0.035308189690113, 0], [0.9888916015625, 0.8079119324684143, 0, 0, 0.4102725386619568, 0, 0], [1.0, 0.546635564416647, 0.9649159908294678, 0, 0, 0, 0], [0, 0.39905660537381965, 0.6326859444379807, 0.0689348056912422, 0.2303346544504165, 0, 0.1327121071517467], [0.9231947958469391, 0.5340053041776022, 0.1690488308668136, 0, 0.9987395405769348, 0, 0.6432285855213801], [0.8562845885753632, 0, 0.1786884516477584, 0.5510889068245888, 0, 0, 0.2550312280654907], [0, 0.5063136617342631, 0.8507179617881775, 0, 0.5647611916065216, 0, 0], [0.9978983402252196, 0.5523485988378525, 0, 0, 0.9821915030479432, 0, 0], [0, 0.3176757246255874, 0.22325392067432398, 0, 0.3558525890111923, 0, 0.1815146654844284], [0, 0.3919810702403386, 0.5349002331495285, 0.44712087512016296, 0.7652405500411987, 0, 0], [0, 0.9978742599487304, 0, 0.9922943115234376, 0.9901090264320374, 0, 0], [0, 0.5994426608085632, 0.4309671223163605, 0, 0, 0, 0], [0.9978166818618774, 0, 0.1292245537042617, 0.8263155966997147, 0, 0, 0], [0, 0.9274757504463196, 0, 0.45805882786711055, 0, 0, 0], [0, 0.2799848228693008, 0, 0.6328828483819962, 0.8793724477291107, 0, 0], [0, 0.3456697314977646, 0.6213160753250122, 0.14388381130993358, 0, 0.3303889334201813, 0], [0, 0.6092732071876525, 0.6408600211143494, 0, 0.5972318798303604, 0, 0], [0, 0.5054946273565293, 0.30851583182811737, 0.6773099005222321, 0, 0, 0], [0.802270770072937, 0, 0, 0.4464475214481354, 0, 0.23464918881654734, 0.7581990957260132], [0, 0.48001097366213796, 0, 0, 0, 0, 0], [0.8419656157493591, 0.5563418753445148, 0.5241226106882094, 0.0792440921068191, 0, 0.0346792861819267, 0], [0, 0.2932811876138051, 0.0592299699783325, 0.2059565037488937, 0.2641794979572296, 0, 0], [0.7979507247606913, 0.4465004950761795, 0, 0.2741715628653764, 0, 0, 0], [0, 0.5791000425815582, 0.1556778848171234, 0, 0, 0.0394679866731166, 0], [0.7704485356807709, 0, 0.9513320326805115, 0.737192710240682, 0.4819197356700897, 0, 0], [0.9781719446182252, 0.5962215065956116, 0, 0.32303112745285034, 0, 0, 0], [0.8827366828918457, 0.3077099621295929, 0.12230298668146128, 0.21348782380421952, 0.4664995148777962, 0.2056480646133422, 0], [0.9893763065338136, 0.8834720452626547, 0, 0.45202524463335675, 0, 0, 0], [0, 0.3214727317293485, 0.4550713499387105, 0.3499137759208679, 0.13750996937354404, 0.087956003844738, 0], [0, 0.4329731166362762, 0.0731195881962776, 0.4842660203576088, 0, 0, 0], [0.9456060826778412, 0.6813242435455322, 0.8390568792819977, 0.122007317841053, 0, 0, 0], [0, 0.265958309173584, 0.9739426374435424, 0.6485097656647364, 0, 0, 0], [0, 0.73462975025177, 0.777972400188446, 0.5994932120665908, 0.835498591264089, 0, 0], [0.9284268021583557, 0.6166851563113076, 0, 0.4593763351440429, 0, 0, 0.9490879774093628], [0, 0.7111337880293528, 0, 0.43853725492954254, 0, 0, 0], [0, 0.1968529919783274, 0.2112407684326172, 0.6509882211685181, 0.1863065958023071, 0, 0], [0, 0.1307744979858398, 0, 0.29106123248736065, 0.9963919520378112, 0, 0], [0.9863648116588593, 0.23971940949559206, 0, 0.14967084676027295, 0.7432439724604288, 0, 0], [0.8062226921319962, 0.3319841573635737, 0.8854308128356934, 0, 0.7867286801338196, 0, 0], [0, 0.6324196060498556, 0, 0.1613005548715591, 0, 0, 0], [0.8679644465446472, 0.33648414909839625, 0, 0.3731407821178436, 0.1517722904682159, 0, 0], [0.979785978794098, 0.19720972329378123, 0, 0.1593278646469116, 0.2375853657722473, 0, 0.0748015940189361], [0, 0.5664648314317068, 0, 0, 0.7428339123725891, 0, 0], [0, 0.6432402729988098, 0.1605892181396484, 0.0702291652560234, 0, 0.293784886598587, 0], [0.8213959336280823, 0.332244873046875, 0.3458787649869919, 0.3156004548072815, 0, 0, 0], [0.8309128681818644, 0.1393556954960028, 0, 0.4541729435324669, 0, 0, 0], [0.8560372591018677, 0, 0.9172278046607972, 0.8311874747276307, 0, 0, 0], [0.9755725860595704, 0.4749549627304077, 0, 0.38686052922691616, 0.5019310712814331, 0, 0], [0, 0.9605554789304733, 0, 0, 0.9690437763929367, 0, 0], [0, 0.41553446277976036, 0.1443416327238083, 0, 0.5721946954727173, 0.5461642146110535, 0], [0, 0.7363198697566986, 0, 0.16078752279281616, 0, 0, 0], [0.99739408493042, 0.6936240345239639, 0.1596115082502365, 0.9764448602994283, 0.872135728597641, 0, 0], [0, 0, 0.9670518636703492, 0.28878501057624817, 0, 0, 0], [0, 0.17779475077986712, 0.6213883757591248, 0.3416204284876585, 0.8504824638366699, 0, 0], [0.8007368743419647, 0.2689311727881431, 0, 0.48373682690518244, 0.0472000166773796, 0, 0], [0.9556874632835388, 0.6609023064374924, 0, 0.19704150408506388, 0, 0, 0], [0, 0.7767488360404968, 0, 0.32496565394103527, 0, 0, 0.8497326374053955], [0.8804509441057841, 0.0156829394400119, 0, 0, 0.8238566517829895, 0, 0], [0.6733457297086716, 0.4426014522711436, 0.7504709959030151, 0.4655395448207855, 0.5211932361125946, 0, 0], [0, 0.5713000893592834, 0.925439178943634, 0, 0.5862599015235901, 0, 0], [0, 0.33422468105951947, 0.5392124056816101, 0.5470330230891705, 0.7335191170374552, 0, 0], [0.899233877658844, 0.7645756999651591, 0.6326782703399658, 0.35950786620378494, 0.34087981283664703, 0.397239625453949, 0.4850168824195862], [0, 0.25545313830176986, 0, 0, 0, 0, 0], [0.9998890161514282, 0.29533302038908005, 0, 0.5821440815925598, 0, 0, 0], [0.3749108016490936, 0.24348757664362586, 0.20523162372410295, 0, 0, 0, 0.6003963351249695], [0.9995214939117432, 0.9995242357254028, 0, 0.8388077616691589, 0.9899723529815674, 0, 0], [0.9972674250602722, 0.7570285797119141, 0.3269025608897209, 0, 0.9994139671325684, 0, 0], [0, 0, 0, 0.22428161718628623, 0, 0.5461642146110535, 0], [0, 0.44836642444133756, 0.22209590859711165, 0, 0, 0, 0], [0, 0.5533560588955879, 0, 0.5092022672295571, 0.5816636085510254, 0, 0], [0.7984632253646851, 0.46863795071840286, 0, 0.9956745505332948, 0.5651941299438477, 0, 0], [0.9982308745384216, 0.7287799417972565, 0.6130238994956017, 0.5971056322256724, 0.1060821935534477, 0, 0], [0, 0.41119364351034154, 0.3150951266288757, 0.1815596781671047, 0, 0.6822801232337952, 0], [0, 0.3347357213497162, 0, 0.4595852792263031, 0.2344984114170074, 0, 0], [0.9998984336853028, 0, 0.7084952592849731, 0.079223483800888, 0.9805095195770264, 0, 0], [0, 0, 0, 0.8125068135559559, 0, 0, 0], [0.9835769534111024, 0.467913955450058, 0.40767569839954376, 0, 0.623910998304685, 0, 0], [0.7439854145050049, 0.8510943651199341, 0, 0.3962196658054988, 0, 0, 0], [0.49916059523820877, 0.8659530878067017, 0.6360049247741699, 0.5824673771858215, 0.5845181345939636, 0, 0.7579456567764282], [0.1811566352844238, 0.7207683722178141, 0.6099719405174255, 0.7091333121061325, 0.2971775233745575, 0.1764267385005951, 0], [0.7922612031300863, 0.19624385610222808, 0.6904131174087524, 0.15270481258630744, 0.8367864191532135, 0.6318855285644531, 0], [0.0865812599658966, 0.3893836960196495, 0.7687567174434662, 0.7240468561649323, 0.8669095039367676, 0, 0.2205751687288284], [0.9969874620437622, 0.6258768935998281, 0, 0, 0.9823463559150696, 0, 0], [0.8118579387664795, 0.8830622434616089, 0.2105603963136673, 0.643403947353363, 0.9464197754859924, 0, 0], [0.6297754049301147, 0, 0, 0, 0, 0.1636343151330948, 0], [0.8564914464950562, 0.666606068611145, 0.2102146918574969, 0.49373987317085266, 0.1862142235040664, 0, 0], [0.7557061314582825, 0.4330809433013201, 0, 0.1005442887544632, 0, 0, 0], [0, 0.26537132759888965, 0, 0.400124525030454, 0.2108171731233596, 0, 0], [0, 0.3123012036085129, 0.3201530873775482, 0.4311063513159752, 0, 0, 0], [0, 0, 0, 0, 0.7805843949317932, 0.6172933330138525, 0], [0, 0, 0, 0.9873892068862916, 0.6170027852058411, 0, 0], [0.6844281256198883, 0.1958197951316833, 0, 0.9949105381965636, 0, 0.936664581298828, 0], [0.7379043400287628, 0.1619765609502792, 0.9908735752105712, 0, 0.48744473854700726, 0, 0], [0.993450164794922, 0, 0, 0.8544389520372663, 0.9306215047836304, 0, 0], [0.6508020162582397, 0.2492445409297943, 0.5772055188814799, 0.3132221270352602, 0, 0, 0], [0, 0.107560709118843, 0, 0.5225786268711089, 0, 0, 0], [0.8143219351768494, 0, 0, 0.5865894675254821, 0, 0, 0], [0.6367020010948181, 0.834637721379598, 0.2138813585042953, 0, 0.20519730634987352, 0, 0], [0, 0.0208768136799335, 0.7924904227256775, 0.18323681503534317, 0.24944016337394714, 0, 0], [0.9942387342453004, 0.8171642422676086, 0.9450928568840027, 0.8407741189002991, 0, 0, 0], [0, 0.5815988692144552, 0, 0, 0.7107625603675842, 0, 0], [0, 0.4518919562300046, 0, 0, 0.3204193413257599, 0, 0], [0, 0.2056365460157394, 0, 0.3455499112606048, 0, 0, 0], [0, 0.8128872875434657, 0, 0.997957706451416, 0.8848416805267334, 0.0570146031677722, 0], [0, 0.420476958155632, 0.5994284600019455, 0.8864354491233826, 0.8968074917793274, 0, 0], [0.816849946975708, 0.38152294357617694, 0, 0, 0.2478728592395782, 0, 0], [0.9950836598873138, 0.9795116583506266, 0, 0, 0.8035786151885986, 0, 0], [0, 0.241003930568695, 0, 0.32489597983658314, 0, 0, 0.2975417375564575], [0, 0.3310004025697708, 0.9985761642456056, 0.2645876109600067, 0.7830245196819305, 0, 0.9954328536987304], [0.7088442742824554, 0.5205787122249603, 0.8420835137367249, 0, 0.9601686000823976, 0.6900169849395752, 0], [0, 0, 0.5729622840881348, 0, 0.9634013175964355, 0, 0], [0.8381215631961823, 0, 0.8683924973011017, 0, 0.9665632843971252, 0.9786466956138612, 0], [0, 0.1856657937169075, 0.1266721487045288, 0.2495082974433899, 0, 0, 0], [0, 0.15945752710103986, 0, 0.5166910650829474, 0, 0, 0], [0.7219779094060262, 0.1588892638683319, 0, 0.17253943532705301, 0.9406763911247252, 0, 0.737669825553894], [0.9934872090816498, 0.7149418294429779, 0.8734255433082581, 0.183787539601326, 0.7730648517608643, 0.0482539385557174, 0], [0.988678514957428, 0.3026688794294993, 0, 0.2336050570011139, 0.35700760781764984, 0, 0], [0, 0, 0.12303000129759309, 0.3007020056247711, 0, 0, 0], [0.9451434016227722, 0.7562824487686157, 0.8175825476646423, 0.5298394560813904, 0, 0, 0.6372030526399612], [0, 0.7905397117137909, 0, 0.7027046233415604, 0.3310720026493072, 0, 0], [0, 0.5621652901172638, 0, 0.8337553143501282, 0.7538435856501261, 0, 0.5987796485424042], [0.8386683464050293, 0.246468611061573, 0, 0.6362367570400238, 0, 0, 0], [0.9572337865829468, 0, 0, 0, 0, 0, 0], [0, 0.7894273996353149, 0.7770558893680573, 0.7630006549879909, 0, 0, 0], [0.5821002125740051, 0.35385642945766443, 0.44948211312294006, 0, 0.5478093586862086, 0.87690669298172, 0], [0.7566160261631012, 0, 0.6336005330085754, 0.5626851618289948, 0.5265098810195923, 0.3148059844970703, 0], [0, 0.1069252975285053, 0.2171537925799687, 0, 0.7322218418121338, 0.099958561360836, 0], [0, 0.1281591951847076, 0.8135654628276825, 0.5783732458949089, 0, 0, 0], [0, 0, 0.7666487097740173, 0.5250461008399725, 0.6989677598079046, 0, 0], [0, 0.41810733638703823, 0, 0.4004928469657898, 0, 0, 0], [0, 0.5602727284034094, 0, 0, 0.0885403081774711, 0, 0], [0, 0.2330359667539596, 0, 0.0880937278270721, 0, 0, 0], [0, 0.4942365121096372, 0, 0.3730760812759399, 0.1829210072755813, 0, 0], [0.9928684234619141, 0.8692396283149719, 0, 0, 0.9359611570835114, 0, 0], [0, 0.9447527527809144, 0.709875762462616, 0, 0.8256950676441193, 0, 0], [0, 0.9781196117401124, 0, 0.6848847766717275, 0, 0, 0], [0, 0.4476963877677917, 0, 0.503273886628449, 0, 0, 0], [0.9608362118403116, 0.5362225882709026, 0.7349990755319595, 0, 0.6177731305360794, 0, 0], [0.9984353184700012, 0, 0.6312538981437683, 0.5850812122225761, 0.918816938996315, 0, 0], [0.9612076282501221, 0.3263781815767288, 0, 0, 0.5791119933128357, 0, 0], [0.9940436482429504, 0.730781614780426, 0.9449558854103088, 0, 0, 0.2127017825841903, 0], [0, 0.6274682333072027, 0, 0.6888274550437927, 0.7873527407646179, 0, 0], [0.8503265182177225, 0.5099145472049713, 0, 0, 0.942871630191803, 0.5583181381225586, 0], [0.7112729748090109, 0.1696243584156036, 0.6324089169502258, 0, 0.9075960516929626, 0.0961633995175361, 0], [0, 0.6598126292228699, 0.6590160727500916, 0.5899926821390787, 0.0334092229604721, 0, 0.4139151275157928], [0.6768554449081421, 0.6652222126722335, 0, 0.5039225518703461, 0.838440865278244, 0, 0.8736395239830017], [0.9745277464389801, 0.7421181797981262, 0, 0, 0, 0, 0], [0.9343262910842896, 0.2946454733610153, 0, 0.5565507039427757, 0.4021599590778351, 0, 0], [0, 0.22429838031530375, 0.5029482841491699, 0.742833286523819, 0, 0, 0], [0.995016872882843, 0.21270858496427533, 0.498596578836441, 0.5551681195696195, 0, 0, 0], [0, 0.6314344257116318, 0.7965166568756104, 0.5211942031979561, 0, 0, 0], [0.7731334865093231, 0, 0.3416183590888977, 0, 0, 0.6009566783905029, 0], [0.959713488817215, 0.6702028065919876, 0.8838085532188416, 0.7258712351322174, 0, 0, 0], [0, 0.08262122608721251, 0, 0, 0.9779844880104064, 0, 0.5467406660318375], [0, 0.978966474533081, 0, 0, 0, 0, 0], [0.1223040595650672, 0.6929327895243963, 0.4569462835788727, 0.7847207188606262, 0, 0, 0], [0.882153848807017, 0.5621363818645477, 0, 0, 0, 0, 0], [0, 0.3586941063404083, 0, 0.381657987833023, 0.5497039929032326, 0, 0], [0, 0.497615497559309, 0, 0, 0.978785514831543, 0, 0], [0.7790702879428864, 0.2149482096234957, 0.3453178480267524, 0.2670140564441681, 0.5723929032683372, 0, 0], [0.6039836406707764, 0.7528332054615021, 0.3052937388420105, 0.6449896395206451, 0, 0, 0.9179275333881378], [0, 0.33526357511679333, 0.6451239585876465, 0.5189908295869827, 0.4281316101551056, 0.22580205649137497, 0], [0.6477344483137131, 0.47609639167785645, 0, 0.5120636895298958, 0, 0.7646875381469727, 0], [0, 0, 0, 0.6064365059137344, 0, 0, 0], [0, 0, 0.5158574655652046, 0.4046141505241394, 0.6972938974698385, 0, 0], [0.5423881337046623, 0, 0.038616694509983, 0, 0.7057843208312988, 0, 0], [0, 0.9402317851781845, 0, 0, 0, 0, 0], [0.8040511906147003, 0.29817384853959084, 0.7806905110677084, 0, 0.7617929577827454, 0, 0.1583788990974426], [0.9550345540046692, 0.6622911840677261, 0.5434426069259644, 0, 0.926654577255249, 0, 0], [0, 0.7678103446960449, 0.8268782496452332, 0.340161383152008, 0.641480565071106, 0, 0], [0, 0.9966151118278505, 0, 0.7715120911598206, 0.975661277770996, 0, 0], [0.7435134053230286, 0.4030219515164693, 0.076835311949253, 0.5595076282819113, 0, 0, 0], [0, 0.8110315799713135, 0.3243432641029358, 0.3337995707988739, 0.9988631010055542, 0, 0.4441543817520141], [0.9996440410614014, 0.852225735783577, 0, 0.9803370833396912, 0, 0, 0], [0.2524967789649963, 0.18975691994031266, 0.5360426306724548, 0, 0.9645208716392516, 0, 0], [0, 0, 0, 0.5206875093281269, 0.1082221716642379, 0, 0], [0, 0.523106706639131, 0, 0.4798218235373497, 0.9025421738624572, 0, 0], [0.7834787964820862, 0.8023123294115067, 0, 0.8881077766418457, 0.6748210787773132, 0, 0], [0, 0.7796804904937744, 0, 0, 0, 0, 0], [0, 0.1838538348674774, 0.4865972697734833, 0.5810543100039164, 0.8425673047701517, 0, 0], [0.953156352043152, 0.5079969167709351, 0, 0.31162513978779316, 0, 0, 0], [0, 0.4157935120165348, 0, 0.0754139795899391, 0, 0.2654641270637512, 0], [0, 0, 0, 0, 0.8711217194795609, 0, 0], [0.8219998478889465, 0.37386333942413325, 0, 0.2543750703334808, 0, 0, 0], [0, 0.1966334730386734, 0, 0, 0, 0, 0], [0.8314670026302338, 0.3530103266239166, 0, 0, 0.2253075391054153, 0, 0], [0, 0, 0.8220926225185394, 0.34634541906416416, 0.3037208914756775, 0, 0], [0.9372914036115011, 0, 0, 0.8409718871116638, 0.5964809060096741, 0, 0], [0, 0.35249607264995575, 0, 0.5359532907605171, 0.6954401135444641, 0.950758934020996, 0], [0, 0.8744927644729614, 0, 0.910670042037964, 0, 0, 0.4472685158252716], [0, 0.910546600818634, 0.1028284523636102, 0, 0.883188009262085, 0.0759967043995857, 0], [0, 0.27520420650641125, 0.2702395021915436, 0.3174309954047203, 0.6858702749013901, 0, 0.4024299085140228], [0.9251145720481873, 0, 0.45697402209043503, 0.3043459951877594, 0, 0, 0.9620466232299804], [0, 0.30384591221809387, 0, 0.3965531438589096, 0, 0, 0], [0, 0.3800164759159088, 0, 0, 0.4251860678195953, 0, 0], [0.9764615297317504, 0.4839383413394292, 0, 0.7046013474464417, 0.3556149899959564, 0, 0], [0, 0, 0.2036140114068985, 0.4817601641019185, 0.152746707201004, 0, 0], [0.9710545539855956, 0.8153036534786224, 0.6385959386825562, 0.3077259063720703, 0.9854795336723328, 0, 0], [0.6438920845588049, 0.24616820116837815, 0, 0.2717242240905761, 0.5915159359574318, 0, 0], [0, 0, 0.6256106495857239, 0.4951717058817546, 0, 0, 0.1851842403411865], [0.7007572650909424, 0, 0, 0.37960462272167206, 0, 0, 0], [0.9278469681739808, 0.2468920946121215, 0, 0, 0.8570576310157776, 0, 0], [0, 0.5871265828609467, 0.7223671674728394, 0.7077268163363138, 0.6723152101039886, 0, 0], [0, 0.5642711073160172, 0.3435225784778595, 0.1312782317399978, 0, 0.3536067903041839, 0], [0.5162562727928162, 0, 0.8392302393913269, 0.7154431939125061, 0, 0, 0], [0.147100567817688, 0.2298146169632673, 0.7731201648712158, 0.1227513402700424, 0.44094385951757425, 0, 0], [0, 0.8266580899556478, 0.6149392127990723, 0, 0.8964510560035706, 0, 0], [0, 0.7314876914024353, 0, 0.4436020515859127, 0, 0, 0], [0, 0.5136096365749836, 0, 0, 0, 0.6927461624145508, 0.140734925866127], [0.6238311479489008, 0.5360317975282669, 0.8143848180770874, 0, 0.8052598237991333, 0, 0], [0.6015934944152832, 0.5281173586845397, 0, 0, 0, 0, 0], [0, 0.6524821780622005, 0, 0.30423002131283283, 0.2955431640148163, 0.3832864761352539, 0], [0, 0.955089032649994, 0, 0, 0.3928476870059967, 0, 0.2172146737575531], [0, 0.3461666417618592, 0, 0.41486856341362, 0.6018030246098837, 0, 0], [0.9369957447052002, 0, 0, 0.1213993784040212, 0, 0, 0], [0, 0, 0, 0.8148401578267416, 0, 0, 0], [0.9768990874290466, 0.5057251542806626, 0, 0.5789487808942795, 0, 0, 0], [0, 0.2821734845638275, 0.3119708299636841, 0, 0.33400761894881725, 0, 0], [0, 0.3140790928155184, 0, 0.9671427011489868, 0.6491116285324097, 0, 0], [0.8002915382385254, 0.461325707534949, 0, 0.1172638138135274, 0.6770708858966827, 0, 0], [0, 0.92870432138443, 0, 0.42716166625420254, 0.906257688999176, 0, 0], [0, 0.47557295858860016, 0, 0.3881077151745558, 0, 0, 0], [0, 0.9891958236694336, 0, 0.9608165621757508, 0, 0, 0], [0.47205908099810284, 0.7996997237205505, 0, 0.31972119584679604, 0.7787186503410339, 0, 0], [0, 0, 0, 0.8683852255344391, 0.7349409461021423, 0, 0], [0.9999998807907104, 0.9777829051017762, 0.9862831234931946, 0.9999991655349731, 0, 0, 0], [0, 0.4845574274659157, 0, 0, 0.0913334637880325, 0, 0], [0.9999896287918092, 0, 0, 0.9777585665384928, 0.9608622193336488, 0, 0], [0, 0.5437105745077133, 0.42627024464309216, 0.07967353239655489, 0.5556866526603699, 0, 0.4145806431770324], [0, 0.4491139352321624, 0.9649006724357604, 0.23558125148216882, 0, 0, 0], [0.9962751865386964, 0.41363291442394257, 0, 0.4216185212135315, 0, 0, 0], [0, 0, 0, 0.6485932171344757, 0, 0, 0], [0.9998890161514282, 0.32824513316154474, 0, 0.2374500781297683, 0, 0, 0], [0, 0.452039768298467, 0.8974469304084778, 0, 0, 0, 0], [0, 0.1993871927261352, 0.0878019407391548, 0.665984034538269, 0.1209334135055542, 0, 0], [0.9384191036224364, 0.1724011376500129, 0.2087767869234085, 0.2537695467472076, 0, 0, 0.7323415875434875], [0.960667073726654, 0.5843788683414459, 0, 0, 0.6256054937839508, 0.1864766180515289, 0], [0, 0.5168917179107666, 0.07351551577448845, 0, 0.6089346607526144, 0, 0], [0, 0, 0, 0.6351051926612854, 0, 0, 0], [0, 0.9111154675483704, 0.930750012397766, 0, 0, 0, 0.9875375628471376], [0, 0.20818388462066645, 0, 0, 0, 0, 0], [0.5600503757596016, 0.0656753927469253, 0, 0, 0.3512012958526611, 0, 0], [0.9953336119651794, 0.3678798079490661, 0.8075927495956421, 0, 0, 0, 0], [0, 0.3053198680281639, 0, 0.24937367439270014, 0, 0.6856358647346497, 0], [0, 0.3840418800711632, 0, 0.2350977826863527, 0, 0, 0], [0, 0.6071517467498779, 0, 0, 0.9803429245948792, 0, 0.5567795038223267], [0, 0.7852000892162323, 0, 0, 0.4911531507968902, 0, 0], [0.95208939909935, 0, 0, 0.6147050062815348, 0, 0.1899990141391754, 0], [0.9926849603652954, 0.343556580444177, 0, 0.555364727973938, 0.2667821049690246, 0, 0], [0, 0.16216929256916046, 0, 0, 0, 0, 0], [0.9573382139205932, 0.7092676758766174, 0, 0.5760601162910461, 0.1828581740458806, 0, 0], [0, 0.9991108775138856, 0, 0.35282890250285465, 0.8010286688804626, 0, 0], [0, 0, 0, 0.2007298022508621, 0, 0, 0], [0, 0.9451653560002645, 0, 0.1803281009197235, 0, 0, 0], [0.9988708794116974, 0.6249582171440125, 0, 0.2539060711860657, 0.9994001388549804, 0, 0.3878496587276459], [0, 0.7544551491737366, 0, 0.6816867798566818, 0.9977295994758606, 0, 0], [0.9029143452644348, 0.3702218234539032, 0, 0.7640600800514221, 0, 0, 0], [0.884465217590332, 0.3094690144062042, 0.6213160753250122, 0.2061213850975036, 0, 0, 0], [0.9810007810592652, 0, 0, 0.0512773990631103, 0.765964150428772, 0, 0], [0, 0.6776813268661499, 0, 0.4881218439550139, 0, 0, 0], [0.8814021348953247, 0.43518852690855664, 0, 0, 0.0283249616622924, 0, 0], [0, 0, 0.5920616909861564, 0.9890457391738892, 0.4395351409912109, 0, 0], [0.8933130204677582, 0, 0, 0.3037295639514923, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0.7404778003692627], [0, 0, 0, 0.4031233787536621, 0.99640154838562, 0, 0], [0, 0.461148053407669, 0, 0.44283581897616386, 0.2058454006910324, 0, 0], [0.9220028221607208, 0, 0.6359447240829468, 0.9592930674552916, 0, 0, 0], [0, 0, 0, 0.4425569511950016, 0, 0, 0], [0.9873189330101012, 0.0060455086641013, 0.8902775347232819, 0, 0, 0, 0], [0.9959232807159424, 0.0386448092758655, 0, 0.7156287531057993, 0, 0, 0], [0, 0, 0.645861804485321, 0.1106426520273089, 0.9364227652549744, 0, 0], [0.129042848944664, 0.47644665837287903, 0, 0.4139335602521896, 0.1252190023660659, 0, 0], [0.5930022597312927, 0.698541522026062, 0, 0.1505200117826461, 0, 0, 0], [0.9998890161514282, 0.7999073266983032, 0, 0.45566015541553495, 0, 0, 0], [0.993931770324707, 0, 0.1695068925619125, 0.6346914768218994, 0.768495500087738, 0, 0], [0, 0.24813329676787058, 0.8885292410850525, 0.8364865183830261, 0, 0, 0], [0, 0.14843906462192533, 0, 0.1401215940713882, 0, 0, 0.2685529291629791], [0.6887931227684021, 0.8852582573890686, 0, 0, 0, 0, 0], [0.8686851859092712, 0.43364093080162996, 0, 0.8242748081684113, 0, 0, 0], [0, 0.6010861285030842, 0, 0.2368294820189476, 0.3374196887016296, 0, 0.2716951370239258], [0.5866780877113342, 0, 0.0362075716257095, 0.10537018378575642, 0, 0, 0], [0, 0.6964412331581115, 0, 0.3112541139125824, 0, 0, 0], [0, 0.11200514435768123, 0, 0.7785935997962952, 0.8017527858416239, 0, 0], [0.7240637540817261, 0, 0, 0.3500118032097816, 0, 0, 0.6677486896514893], [0.9916799664497375, 0.7147151529788971, 0.7235030879576999, 0, 0.9466767311096191, 0, 0], [0, 0.1857205629348755, 0.6096715927124023, 0, 0, 0, 0], [0, 0.606584203739961, 0, 0.4164561927318573, 0.6092907190322876, 0, 0], [0.9979648590087892, 0, 0.1264958977699279, 0.3726506332556407, 0, 0, 0], [0, 0.5704122185707091, 0, 0.5717292129993439, 0, 0, 0], [0, 0, 0.9629878997802734, 0.9613381624221802, 0.8398213187853495, 0, 0], [0, 0.6706158071756363, 0, 0, 0.9973130822181702, 0, 0], [0.988421638806661, 0, 0.9802175760269164, 0.8149416049321493, 0, 0, 0], [0, 0.37328919768333435, 0.3806568682193756, 0.1496919989585876, 0.3341141045093536, 0, 0], [0, 0.9679225087165833, 0.8483239710330963, 0.6074412862459818, 0, 0, 0], [0.8902291456858317, 0, 0, 0.26659453660249705, 0, 0.7322906255722046, 0.9686283469200134], [0.9999173879623412, 0.6506758630275726, 0, 0, 0, 0, 0.9089216589927672], [0.72590771317482, 0.4005575142800808, 0, 0.0942002162337303, 0, 0, 0], [0.9852933287620544, 0.6502009630203247, 0.5659034252166748, 0.9324586987495422, 0, 0, 0], [0, 0, 0, 0.3443641662597656, 0, 0, 0], [0.9946546256542206, 0, 0, 0, 0, 0, 0], [0.9746590852737428, 0.3960438966751098, 0, 0.6034130652745565, 0, 0, 0], [0, 0.4194024329384168, 0.5128115545958281, 0, 0.999326229095459, 0, 0], [0.8137806951999664, 0.3357343425353368, 0, 0, 0.8566827178001404, 0.9168493747711182, 0], [0.9663036465644836, 0.8545006364583969, 0, 0, 0.8577377398808798, 0, 0], [0.9630252718925476, 0, 0, 0.18535096943378446, 0, 0, 0], [0.9080576598644257, 0, 0, 0.9467307925224304, 0, 0, 0.9354699850082396], [0.997933268547058, 0.2280149273574352, 0.7801306843757629, 0.6138735711574554, 0, 0.7431475520133972, 0], [0, 0.09757677714029944, 0.406447172164917, 0.5127186179161072, 0, 0, 0], [0.9999897480010986, 0, 0, 0.07559471460990601, 0, 0, 0], [0.966854214668274, 0.3062873780727386, 0.2078906148672104, 0, 0, 0.919008731842041, 0.7302050590515137], [0, 0, 0.0893199518322944, 0, 0.5291503667831421, 0, 0], [0.7353353798389435, 0, 0, 0.3155672922730446, 0, 0, 0], [0, 0.3503738343715668, 0.9473413228988647, 0.2772431564517319, 0.2051386684179306, 0.3008444905281067, 0], [0, 0, 0.9513964653015136, 0.3079771399497986, 0.3163562417030334, 0, 0], [0, 0.9999984502792358, 0, 0, 0.1739282757043838, 0, 0], [0, 0.7475153803825378, 0, 0, 0.8519415855407715, 0.5209540128707886, 0.9566513895988464], [0, 0, 0.404523104429245, 0.0895131975412368, 0.2296401411294937, 0, 0], [0, 0.6387652605772018, 0.5360426306724548, 0.5632560551166534, 0, 0, 0], [0, 0.39628566801548004, 0, 0, 0, 0, 0.976010799407959], [0, 0.22181129455566406, 0.38981058448553085, 0.5897918492555618, 0, 0, 0], [0.6972024738788605, 0.9975630044937134, 0.2032715231180191, 0.9474552869796752, 0, 0, 0.9050750732421876], [0, 0, 0, 0.6985581815242767, 0, 0, 0], [0.9976856708526612, 0.6871193647384644, 0, 0.39468874658147496, 0, 0, 0], [0.9470542669296265, 0.8392125248908997, 0.983055830001831, 0, 0, 0, 0], [0, 0.1830735206604004, 0, 0, 0, 0, 0.5817651748657227], [0, 0.1763667315244674, 0, 0.39093559887260193, 0, 0, 0], [0, 0.9086764752864838, 0, 0.6986735016107559, 0, 0, 0], [0, 0.9401506781578064, 0, 0.5750368535518646, 0, 0, 0], [0, 0, 0.15843986868858337, 0, 0.447942520181338, 0, 0], [0, 0.677143007516861, 0, 0.3245876729488373, 0, 0, 0], [0, 0.580712248881658, 0.4119803011417389, 0.16173540242016313, 0.5180255174636841, 0, 0], [0, 0, 0.0722702071070671, 0.6277021765708923, 0.5686312913894653, 0, 0], [0, 0.3680701442062855, 0.7937846183776855, 0, 0, 0, 0], [0, 0, 0, 0.1726616670687993, 0.6385231614112854, 0.7106927037239075, 0], [0, 0.6990181356668472, 0.986528754234314, 0.9908964335918427, 0, 0.280539333820343, 0], [0, 0.712792177995046, 0, 0.3012395873665809, 0, 0, 0], [0, 0.3782924506813288, 0, 0, 0, 0, 0], [0.9637864232063292, 0.5199175501863161, 0.8809828758239746, 0, 0, 0, 0], [0.727125883102417, 0.2432291656732559, 0, 0, 0.4630696177482605, 0, 0], [0, 0, 0.934254825115204, 0, 0, 0, 0], [0.9914891719818116, 0.0865883007645607, 0, 0.2399685382843017, 0, 0, 0], [0.8761524558067322, 0, 0.2047822922468185, 0.19045627117156982, 0, 0, 0], [0, 0.47995344921946526, 0, 0, 0, 0, 0], [0, 0.5836714108784994, 0, 0.3922363929450512, 0, 0, 0], [0.8075020909309387, 0.7730647325515747, 0, 0.968628227710724, 0, 0, 0], [0.9311657547950744, 0.943433403968811, 0, 0.5910688415169716, 0, 0, 0], [0, 0, 0.6156893670558929, 0, 0, 0, 0], [0.5492483377456665, 0.3685123473405838, 0, 0.975142776966095, 0, 0, 0], [0, 0.22468116879463193, 0.3357025384902954, 0, 0.46190613601356745, 0, 0], [0.2103320062160492, 0.510488490263621, 0, 0, 0.7076171040534973, 0, 0], [0, 0.7530733346939087, 0, 0, 0.5821109190583229, 0, 0], [0.9887515902519226, 0.8558133244514465, 0, 0, 0.4675808846950531, 0, 0.895248293876648], [0, 0.5724770203232765, 0, 0.9999996423721312, 0, 0, 0], [0, 0.1109719574451446, 0.5588555335998535, 0, 0.6349553018808365, 0.4018196165561676, 0.2173197120428085], [0, 0.0444766990840435, 0, 0.2880449593067169, 0, 0, 0.755434513092041], [0.9753471612930298, 0.1466142833232879, 0, 0.5031121894717216, 0, 0, 0.37667442858219147], [0.68071049451828, 0.821828305721283, 0, 0.13692355714738363, 0, 0, 0], [0, 0.3579537719488144, 0.6637248024344444, 0.9874749779701232, 0, 0, 0], [0.8901060819625854, 0.15625682473182673, 0.2010165303945541, 0, 0, 0, 0], [0, 0.7510431607564291, 0, 0.9547904133796692, 0, 0, 0], [0.8836984038352966, 0.6796407699584961, 0, 0, 0, 0, 0], [0, 0.44729113578796387, 0, 0.6733046770095825, 0, 0, 0], [0.975016951560974, 0.1876919567584991, 0, 0.5496038943529129, 0, 0, 0], [0, 0.3202250227332115, 0, 0.571772076189518, 0.1467051804065704, 0, 0], [0.4290705323219299, 0, 0.8583927750587463, 0, 0.1655242592096328, 0, 0.8862242698669434], [0, 0.0584130361676216, 0, 0.6802905891090631, 0, 0.4347957372665405, 0], [0, 0, 0.384603351354599, 0.349162923792998, 0, 0, 0], [0.8730899691581726, 0.1734088957309723, 0, 0.5663340166211128, 0.3764902502298355, 0, 0], [0, 0.9222001135349274, 0.3234823644161224, 0.4281940460205078, 0.9070896506309508, 0, 0], [0, 0, 0, 0.4410122036933899, 0, 0, 0], [0.8346452713012695, 0.1967104971408844, 0, 0.2859788089990616, 0.9323501586914062, 0, 0], [0.8690880338350931, 0, 0, 0, 0, 0.0839934423565864, 0], [0, 0, 0, 0.4174357950687408, 0.8870745301246643, 0, 0], [0, 0.1058510914444923, 0, 0.2047339826822281, 0.7394542098045349, 0, 0], [0, 0.6210158169269562, 0.1737950146198272, 0.23094109197457627, 0.9999322891235352, 0, 0], [0.8784289360046387, 0.3039485563834508, 0, 0.2875932529568672, 0, 0, 0.7574154138565063], [0, 0, 0, 0.28843718270460766, 0.7088857889175415, 0, 0], [0, 0.5631498346726099, 0, 0, 0.515790268778801, 0, 0], [0, 0.6232825070619583, 0, 0.5245604068040848, 0.9671982526779176, 0, 0], [0.9419534802436828, 0.1688170880079269, 0, 0.3775460471709569, 0, 0, 0.1915683895349502], [0.9919490814208984, 0.2060367614030838, 0, 0, 0, 0, 0], [0, 0.5886640052000681, 0, 0.7947491010030111, 0, 0, 0], [0, 0.7619930803775787, 0, 0.15058701485395432, 0.5990972518920898, 0, 0], [0, 0, 0, 0.28868946433067316, 0, 0, 0], [0.999836564064026, 0.9977853894233704, 0, 0.1932415962219238, 0, 0, 0], [0.9725577533245087, 0, 0, 0, 0, 0, 0], [0.7887290716171265, 0.9011455178260804, 0, 0.5838282306989034, 0, 0, 0.2373962700366974], [0, 0.8586428761482239, 0, 0, 0.982642650604248, 0, 0], [0, 0, 0, 0.4405561722815037, 0, 0, 0], [0, 0.701813280582428, 0, 0.5304955095052718, 0, 0, 0], [0, 0.572505310177803, 0, 0.1346357762813568, 0, 0, 0], [0.855043351650238, 0.9763086438179016, 0.8073421120643616, 0.26600638404488564, 0.8395379781723022, 0, 0], [0, 0.5962350964546204, 0.8225999176502228, 0, 0, 0, 0], [0.58710777759552, 0.6414648294448853, 0, 0.4321666806936264, 0.8989986181259155, 0, 0], [0, 0, 0, 0.5618552193045616, 0.9931278228759766, 0, 0], [0.852134644985199, 0.193113163113594, 0, 0.1132744774222374, 0, 0, 0], [0.8500884175300598, 0.35060690840085346, 0.964422345161438, 0, 0.7344199419021606, 0, 0], [0, 0.7404531836509705, 0, 0.7889294475317001, 0, 0, 0], [0.8631029923756918, 0.3579963967204094, 0, 0, 0, 0, 0], [0.5425298810005188, 0.42797674238681793, 0, 0.5212516188621521, 0.2561501860618591, 0, 0], [0, 0.8806353211402893, 0, 0.8989295164744059, 0, 0, 0], [0, 0.7442100048065186, 0, 0.29551886518796283, 0.5291455388069153, 0, 0], [0.9933944940567017, 0.5130248218774796, 0.0070564872585237, 0.6388458013534546, 0.5870093703269958, 0, 0.998559296131134], [0.4586570560932159, 0.5797359123826026, 0, 0.7065043449401855, 0, 0.9606325030326844, 0], [0, 0, 0, 0.0298800654709339, 0, 0, 0], [0.9985498785972596, 0.8576722741127014, 0.2047391384840011, 0, 0.5726414620876312, 0, 0], [0, 0.2717212438583374, 0.7099677920341492, 0.7404987514019012, 0.9853609204292296, 0, 0], [0, 0.38433055579662323, 0.9606517553329468, 0.1980326175689697, 0.0018733113538473, 0, 0], [0.4756860733032226, 0.9405284523963928, 0, 0.967712938785553, 0, 0, 0], [0, 0.9996680021286012, 0, 0.8830722570419312, 0.6615848541259766, 0.7329308390617371, 0], [0, 0.2166718542575836, 0, 0.5439864993095398, 0, 0, 0], [0.7422663569450378, 0.21673345565795896, 0.7736464738845825, 0, 0, 0, 0], [0.9613361954689026, 0, 0, 0.0967212840914726, 0, 0.2987103462219238, 0], [0, 0.1627363413572311, 0.22793797403573984, 0, 0.1728652417659759, 0, 0], [0.85548335313797, 0.9657625555992126, 0.0875521004199981, 0.8676820000012716, 0, 0, 0], [0.7478852272033691, 0.40291969776153563, 0.7863990068435669, 0.4353816509246826, 0, 0, 0], [0.7881165146827698, 0.1376917064189911, 0, 0, 0, 0, 0], [0.5237802565097809, 0, 0, 0.1531589776277542, 0, 0, 0], [0.9227574467658995, 0.6388328075408936, 0, 0.9583064913749696, 0, 0, 0], [0, 0.4851929247379303, 0.5078431963920593, 0.996380627155304, 0.9973886609077454, 0, 0], [0.7543229758739471, 0.4663476049900055, 0.8870601058006287, 0, 0, 0, 0], [0.8350342512130737, 0.6160757094621658, 0.47502748668193817, 0, 0.4402572214603424, 0, 0], [0, 0.24915181100368497, 0.939636766910553, 0.1481101214885711, 0.48844029754400253, 0, 0], [0.7550103068351746, 0, 0, 0.7853804230690002, 0.8235424757003784, 0, 0], [0, 0, 0, 0.5348197221755981, 0, 0, 0], [0.1941781342029571, 0, 0.8209009766578674, 0.2677595913410187, 0, 0, 0], [0.806919515132904, 0, 0.0683556199073791, 0, 0, 0, 0], [0, 0.185909554362297, 0, 0.8055852850278219, 0, 0, 0], [0.9983119964599608, 0.9582659006118774, 0, 0.5786018744111061, 0, 0, 0], [0, 0.6757785946130752, 0.6325847506523132, 0.1677526533603668, 0, 0, 0], [0, 0.2769373804330826, 0, 0, 0, 0, 0.1732809245586395], [0.9066005746523539, 0, 0, 0, 0, 0, 0], [0, 0, 0.2032179534435272, 0.7523135393857956, 0.9789738655090332, 0, 0], [0, 0, 0.1818059906363487, 0, 0.8074188530445099, 0, 0], [0, 0.16265640258789055, 0.41434724628925323, 0, 0, 0, 0], [0, 0.41386882308870554, 0, 0.2464707543452581, 0, 0, 0], [0, 0.941849946975708, 0, 0.9994484782218932, 0, 0, 0.6391354203224182], [0, 0.2578998804092407, 0, 0.23977633193135262, 0.8098289966583252, 0, 0], [0.9919453859329224, 0, 0, 0.5864589810371399, 0, 0, 0], [0, 0.2194991111755371, 0, 0.5761193633079529, 0.9166260957717896, 0, 0], [0, 0, 0, 0.9899231791496276, 0, 0, 0], [0, 0, 0, 0.13992113806307316, 0, 0, 0], [0, 0.9023181398709615, 0, 0.9544405043125153, 0.8746390342712402, 0, 0], [0, 0.5993314683437347, 0.2393578290939331, 0.4413269162178039, 0, 0.3032544255256653, 0], [0, 0.5587488412857056, 0.327093094587326, 0.7474697828292847, 0, 0, 0], [0.9542343020439148, 0.8732752501964569, 0.7260780930519104, 0, 0.7888891696929932, 0, 0], [0.9930824041366576, 0.5781004726886749, 0, 0.19251259416341782, 0, 0, 0], [0, 0, 0.6722255945205688, 0.48293141027291614, 0, 0.0661607608199119, 0], [0.9962896307309469, 0.6108551422754923, 0, 0, 0, 0, 0], [0, 0.5478115528821945, 0, 0, 0.2585894465446472, 0.7577463388442993, 0], [0, 0.5244430899620056, 0, 0, 0, 0.7778548002243042, 0], [0, 0.9031608700752258, 0, 0.6722092827161154, 0, 0, 0], [0, 0.5359324067831039, 0.9795721173286438, 0, 0, 0, 0], [0, 0.9085646867752076, 0, 0.5590359196066856, 0.5553073063492775, 0, 0], [0, 0, 0.3049352864424388, 0.4335543115933736, 0, 0, 0], [0.9655671715736388, 0.187717318534851, 0, 0.8277181386947632, 0, 0, 0], [0.4928815364837646, 0.1938001712163289, 0.0878019407391548, 0, 0.8594896793365479, 0.0833527296781539, 0], [0, 0, 0, 0.7190126180648804, 0, 0, 0], [0, 0.693662424882253, 0, 0, 0, 0, 0], [0.9396730065345764, 0.6671939492225647, 0, 0.4351479560136795, 0, 0, 0], [0.8580000400543213, 0.3792301416397095, 0, 0.1384925171732902, 0, 0.762707531452179, 0], [0, 0.3337550163269043, 0, 0.8392081409692764, 0.8199997544288635, 0, 0], [0.2769265472888946, 0.4647690951824188, 0, 0.8350702524185181, 0, 0, 0], [0, 0.4238203763961792, 0.4119803011417389, 0.12511056195944545, 0.7204829454421997, 0, 0], [0.2009294182062149, 0.1401687413454055, 0.1341946125030517, 0.33011963218450546, 0, 0, 0], [0.9856629967689514, 0.5685001760721207, 0, 0, 0, 0, 0], [0.965239405632019, 0, 0.864943265914917, 0.24612613767385477, 0, 0, 0], [0, 0.4858341217041015, 0, 0.2506068646907806, 0.7429352402687073, 0, 0], [0, 0.5707161575555801, 0, 0.7039675116539001, 0.9670658707618712, 0.7457398772239685, 0], [0, 0, 0, 0.2697717547416687, 0, 0, 0], [0, 0, 0.4910050332546234, 0, 0.7110612988471985, 0, 0], [0.6488620638847351, 0.685554027557373, 0, 0.5151886940002441, 0, 0, 0], [0, 0, 0.0732994824647903, 0.1869460195302963, 0, 0, 0], [0, 0.2229778170585632, 0, 0, 0.6201033592224121, 0.2956096231937408, 0], [0, 0, 0, 0, 0.37567200511693954, 0, 0], [0, 0.9746620655059814, 0.1282423138618469, 0, 0.7275846004486084, 0, 0], [0, 0.5618903736273447, 0, 0.2859173417091369, 0, 0, 0], [0, 0, 0, 0.736616313457489, 0, 0, 0], [0, 0.7650484442710876, 0, 0, 0, 0, 0], [0, 0.44517527520656586, 0, 0.3837544918060303, 0, 0, 0], [0, 0.2042194902896881, 0, 0.9999996423721312, 0, 0, 0], [0.8275470435619354, 0.8329088091850281, 0, 0, 0.7790562212467194, 0.0180742442607879, 0.2330736517906189], [0, 0.1650880724191665, 0, 0, 0.4077780246734619, 0, 0], [0, 0.9469757080078124, 0.39196640253067017, 0, 0.3988919258117676, 0, 0], [0, 0.3864554166793823, 0.8111147284507751, 0.3964209146797657, 0, 0, 0], [0.7305022925138474, 0.3911655694246292, 0, 0.2145054787397384, 0.7364519238471985, 0.494967371225357, 0], [0, 0, 0.7642458081245422, 0.6462923914194107, 0, 0, 0], [0.9435834884643555, 0.6614264696836472, 0.6317604680856069, 0.2246100455522537, 0, 0, 0], [0.8157311677932739, 0, 0, 0.7235521078109741, 0.9664013385772704, 0, 0], [0, 0, 0.3055712878704071, 0, 0.6060523986816406, 0, 0], [0.8975757360458374, 0.24436493590474126, 0, 0.5727054923772812, 0, 0, 0], [0, 0, 0, 0.1169563345611095, 0, 0, 0], [0, 0.2989444211125374, 0, 0.24394591152667994, 0, 0, 0], [0.9988512992858888, 0, 0, 0.9636210203170776, 0, 0, 0], [0, 0.9999988873799642, 0, 0, 0, 0, 0], [0, 0, 0, 0.3065113474925359, 0, 0, 0], [0.9993160963058472, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0.1874303966760635, 0.490555852651596, 0, 0], [0, 0, 0, 0.5091973543167114, 0.4614312648773193, 0, 0], [0.7321107983589172, 0.4378501698374748, 0, 0, 0, 0, 0], [0.97362220287323, 0.24818605184555054, 0, 0, 0, 0, 0], [0, 0.1812153309583664, 0, 0.658309648434321, 0, 0, 0.2665957510471344], [0.335129827260971, 0.6240821182727814, 0.5882979109883308, 0, 0, 0, 0], [0, 0.8018253445625305, 0, 0.8372848033905029, 0.9061606228351593, 0.0847030505537986, 0], [0, 0.7201951742172241, 0.9261971414089203, 0, 0.9654997885227203, 0, 0], [0, 0.8262170553207397, 0, 0.7899132072925568, 0, 0, 0], [0, 0.619205892086029, 0, 0.7338870763778687, 0.7658072113990784, 0, 0], [0, 0, 0.8282636404037476, 0.4825395345687866, 0, 0, 0], [0, 0, 0, 0.929259955883026, 0, 0, 0], [0, 0, 0.2409137934446334, 0.1509837955236435, 0, 0, 0], [0.4535481929779053, 0, 0.2210901826620102, 0.4547971189022064, 0, 0, 0], [0.6479278405507406, 0, 0.3820513486862182, 0, 0.8259788155555725, 0, 0], [0, 0.7567852834860483, 0, 0.7087427973747253, 0, 0, 0], [0.997560739517212, 0, 0, 0.6582297086715698, 0.5875887274742126, 0, 0], [0, 0, 0, 0.7728211085001627, 0, 0, 0], [0, 0.2485635578632354, 0.2658907771110534, 0.1572520732879638, 0, 0, 0], [0, 0.5149025470018387, 0, 0.9478759169578552, 0, 0, 0], [0.9961187839508056, 0, 0, 0, 0.2344469502568245, 0, 0], [0.9705955386161804, 0.955918788909912, 0, 0, 0.9029898047447203, 0, 0], [0, 0, 0, 0.6775863369305929, 0, 0, 0], [0, 0.7591628432273865, 0, 0.127221331000328, 0, 0, 0], [0.7110794186592102, 0.680536150932312, 0, 0.7004865407943726, 0.1321019381284713, 0, 0.5583259463310242], [0, 0.771450936794281, 0.9231293201446532, 0.0858028084039688, 0, 0, 0.9479941725730896], [0, 0.5506677590310574, 0, 0, 0.6888405680656433, 0, 0], [0.6529971659183502, 0.937824785709381, 0, 0.0750802159309387, 0.2313665226101875, 0, 0], [0.9834801554679872, 0, 0, 0.6326945200562477, 0, 0, 0], [0, 0.5678587555885315, 0, 0.17532622814178467, 0.484655037522316, 0, 0], [0, 0, 0, 0.32703647017478943, 0, 0, 0], [0, 0, 0, 0.2737424820661545, 0, 0, 0], [0, 0.937646746635437, 0, 0.8837233781814575, 0, 0, 0], [0, 0, 0.9485570788383484, 0, 0.1020479425787925, 0, 0], [0, 0.35078610479831696, 0, 0.5737656131386757, 0, 0, 0.1929570734500885], [0, 0, 0, 0.6185382902622223, 0, 0, 0], [0.9589570462703706, 0, 0, 0.7784577012062073, 0.6921831369400024, 0, 0], [0, 0.46299326717853545, 0, 0.1336478590965271, 0, 0, 0], [0.8685146421194077, 0.7723398605982462, 0, 0, 0.75841224193573, 0, 0], [0, 0, 0.8820732831954956, 0.461660772562027, 0, 0, 0], [0, 0.1851717382669448, 0, 1.0, 0.9781557321548462, 0, 0.7830358147621155], [0.9826825857162476, 0.9704091548919678, 0, 0.563935711979866, 0, 0, 0], [0, 0.0974400043487548, 0, 0, 0.5456193089485168, 0, 0], [0, 0.1933234147727489, 0, 0.9050100445747375, 0, 0, 0], [0, 0.7483464876810709, 0.8457674980163574, 0.7465699315071106, 0, 0, 0], [0.9038006365299225, 0.3328661918640136, 0, 0.5911608934402466, 0.4549315050244331, 0, 0], [0, 0.7407014966011047, 0, 0.5732750296592711, 0, 0, 0], [0, 0.4474040796048939, 0, 0.1447875276207924, 0, 0, 0], [0, 0.4382002651691437, 0, 0.2063779979944229, 0, 0, 0], [0, 0.4361404553055763, 0, 0, 0, 0, 0], [0, 0.5319429636001587, 0, 0, 0.5433284938335419, 0, 0], [0.9301759004592896, 0.9433960914611816, 0.3798804084459941, 0, 0.1455951482057571, 0, 0], [0, 0.2643985450267792, 0.1730610579252243, 0, 0, 0, 0], [0, 0, 0, 0.9145642638206481, 0, 0, 0], [0.9985268115997314, 0.6000422785679499, 0, 0.996598184108734, 0, 0.0335448645055294, 0], [0, 0.4569946179787318, 0.5880762040615082, 0, 0, 0, 0], [0, 0.8935765624046326, 0, 0, 0, 0, 0], [0, 0.8382226228713989, 0.9002190828323364, 0.9824780821800232, 0, 0, 0], [0.9947651624679564, 0, 0.3537010550498962, 0.2705543637275696, 0, 0.7972285151481628, 0], [0.919966995716095, 0.4640173465013504, 0, 0, 0, 0, 0], [0, 0, 0, 0.9810948371887208, 0, 0, 0], [0, 0.5346655771136284, 0, 0.1866682320833206, 0, 0, 0], [0, 0.1857891976833343, 0, 0.9999996423721312, 0, 0, 0], [0.9850815534591676, 0, 0, 0, 0, 0, 0], [0, 0.1264423262327909, 0.2143518179655075, 0.1607735604047775, 0, 0, 0], [0, 0.6305590867996216, 0, 0.7460231900215148, 0, 0.9326297640800476, 0], [0.7780826687812805, 0.5265422612428665, 0, 0, 0, 0, 0.9470831155776978], [0.9963617523511251, 0, 0, 0.9729297161102295, 0, 0, 0], [0, 0.818442165851593, 0.0355157554149627, 0, 0.9288597106933594, 0, 0.9875763654708862], [0, 0.32258214553197223, 0, 0, 0.3297072052955627, 0, 0], [0, 0.1463252902030944, 0, 0.3490307629108429, 0.8519936800003052, 0, 0], [0.5140544846653938, 0.2713688015937805, 0, 0, 0.5669681429862976, 0, 0], [0.2421191185712814, 0.9376945793628693, 0.9879288077354432, 0.22031734511256212, 0, 0, 0], [0, 0.2101299613714218, 0, 0, 0, 0, 0], [0, 0, 0.9926453828811646, 0.3753054738044739, 0, 0, 0], [0.9788824319839478, 0, 0, 0, 0, 0, 0], [0, 0.4360128790140152, 0.0786635503172874, 0, 0.8131541013717651, 0, 0], [0, 0.6673983931541443, 0, 0.6826788783073425, 0.9965460896492004, 0, 0], [0.7375098764896393, 0.17530412971973414, 0, 0, 0, 0.1116907000541687, 0], [0, 0.6798208951950073, 0, 0.9186642169952393, 0.882199764251709, 0, 0], [0, 0, 0, 0, 0.9744198322296144, 0, 0], [0, 0.5180510655045509, 0, 0.5271496176719666, 0, 0.1489452272653579, 0], [0, 0.24346860249837235, 0, 0.0954240560531616, 0, 0, 0], [0.5937815308570862, 0, 0.4605516791343689, 0.368699312210083, 0, 0.40830525755882263, 0], [0, 0.2796841561794281, 0, 0, 0.4911365807056427, 0, 0], [0.7459213733673096, 0.2256866991519928, 0, 0, 0, 0, 0], [0.213696002960205, 0.4192092716693878, 0, 0.0761165618896484, 0, 0, 0], [0, 0.7398661375045776, 0, 0.551514595746994, 0.5393770933151245, 0, 0], [0, 0, 0.2122107595205307, 0.1820798814296722, 0, 0, 0], [0, 0.4836683012545109, 0, 0.7505519688129425, 0, 0, 0], [0.9999179840087892, 0.5999712273478508, 0, 0, 0, 0, 0], [0.8032838503519694, 0, 0.4345232844352722, 0, 0.0651611536741256, 0, 0], [0, 0.4768611093362172, 0, 0.1991864169637362, 0.9044187664985656, 0, 0], [0, 0, 0, 0.21670018136501304, 0.5198571979999542, 0, 0], [0.8426806628704071, 0.6201927661895752, 0, 0, 0, 0, 0], [0, 0.2593645751476288, 0, 0.9821004271507264, 0, 0, 0], [0, 0.982747495174408, 0.9602136611938475, 0, 0.9433811902999878, 0, 0], [0.9076706767082214, 0, 0, 0.66035957634449, 0.7630181908607483, 0, 0], [0, 0.9998528957366944, 0, 0, 0, 0, 0], [0, 0.4077863196531932, 0, 0, 0, 0, 0], [0.7787826458613077, 0.917360246181488, 0, 0, 0.9904737869898478, 0, 0], [0, 0, 0.1877351701259613, 0, 0.3107549250125885, 0, 0], [0, 0, 0, 0.2439068257808685, 0, 0, 0], [0.6602874398231506, 0.8078694343566895, 0, 0, 0, 0, 0], [0, 0, 0.2050797790288925, 0, 0.3017663657665252, 0, 0], [0, 0.1731486320495605, 0, 0.40965812280774117, 0, 0, 0], [0, 0.7061854004859924, 0, 0.4897054433822632, 0, 0, 0], [0, 0, 0, 0, 0.6699798107147217, 0, 0.037310604006052], [0, 0.2862441688776016, 0.3595884405076504, 0.673895001411438, 0.965898334980011, 0, 0], [0, 0, 0.6258012652397156, 0, 0, 0, 0], [0, 0.962145745754242, 0, 0, 0.2287657409906387, 0, 0], [0.8609865307807922, 0, 0, 0, 0, 0, 0], [0.8553982973098755, 0, 0, 0, 0.3633233606815338, 0, 0], [0.2636506855487823, 0.6212840874989828, 0, 0.9239733219146729, 0.8916575312614441, 0, 0], [0, 0.38421063125133514, 0, 0.17695020760099087, 0, 0, 0], [0.9912809729576112, 0.9999048709869384, 0, 0.9207552075386047, 0.9940647482872008, 0, 0], [0, 0.9270152449607848, 0, 0.1730960756540298, 0, 0, 0.8004189729690552], [0, 0.3977418579161167, 0, 0.5212080975373586, 0.2690344154834747, 0, 0], [0.8221134543418884, 0.7325722724199295, 0.6481255739927291, 0.1682657152414322, 0, 0, 0], [0, 0.9971451163291932, 0, 0, 0, 0, 0], [0, 0.3133101001381874, 0, 0.9310728311538696, 0, 0, 0], [0, 0, 0, 0.3382214456796646, 0, 0.6580610871315002, 0], [0, 0.5439648628234863, 0, 0.34829632317026454, 0, 0, 0], [0, 0, 0, 0.922444224357605, 0, 0, 0], [0.50998904556036, 0.3779731690883636, 0, 0, 0, 0, 0], [0, 0.9999864101409912, 0, 0.9614586234092712, 0.9924091100692748, 0, 0.9981535077095032], [0, 0.577920138835907, 0, 0.9898788928985596, 0.9836265742778778, 0, 0], [0.783454030752182, 0.6939151138067245, 0.7898792624473572, 0.24179781973361963, 0, 0, 0], [0, 0.4286711911360423, 0, 0.605860710144043, 0, 0, 0], [0.7800664901733398, 0.1628996878862381, 0.4564109109342098, 0, 0.3838036656379699, 0, 0], [0, 0.6790453506012758, 0, 0.9905319213867188, 0, 0, 0], [0, 0, 0.3946719169616699, 0.6178484658400217, 0, 0, 0], [0, 0.8250210881233215, 0, 0, 0.7050862312316895, 0, 0], [0, 0.5433710813522339, 0.0554536581039428, 0.535403236746788, 0, 0, 0], [0, 0.9568869471549988, 0.8602281212806702, 0.7569817304611206, 0, 0, 0], [0, 0, 0.2121429592370987, 0.0983974933624267, 0.9478491544723512, 0, 0], [0.9482046365737916, 0.0600416213274002, 0.0878019407391548, 0, 0.6191686689853668, 0, 0], [0, 0.2743957017858823, 0, 0.5266064405441284, 0.6893755197525024, 0, 0.0138356192037463], [0, 0.1012224182486534, 0.9265624284744264, 0, 0, 0, 0], [0.9762431979179382, 0, 0, 0.32179225981235504, 0, 0, 0], [0, 0.6158140500386556, 0.8969684839248657, 0.6303912252187729, 0.8162943720817566, 0, 0], [0.9346108436584472, 0, 0, 0.7735064625740051, 0, 0, 0], [0.8939879536628723, 0.3849499747157097, 0, 0, 0.1922998875379562, 0, 0], [0, 0.48000309616327286, 0, 0.9997653365135192, 0, 0, 0.3053556382656097], [0, 0.939242959022522, 0, 0, 0, 0, 0], [0, 0.4210429787635803, 0.5537383556365967, 0.6818510293960571, 0, 0, 0], [0, 0.05564752686768765, 0, 0, 0, 0, 0], [0, 0.24095069617033, 0, 0.3036436438560486, 0, 0, 0.1075439527630806], [0.7123711705207825, 0.2285552620887756, 0.2530393600463867, 0.0988438427448272, 0, 0, 0], [0.9999996423721312, 0.997621476650238, 0, 0.9636498987674713, 0, 0, 0], [0.6475989073514938, 0.4284641444683075, 0, 0.8478533029556274, 0.9540753364562988, 0, 0], [0, 0, 0, 0.3189812898635864, 0, 0, 0], [0, 0.5986007899045944, 0, 0.5123546347022057, 0.2957467138767242, 0, 0], [0, 0.6675703525543213, 0, 0, 0.3049703538417816, 0, 0], [0, 0.9224998354911804, 0, 0.6399284303188324, 0.3464513123035431, 0, 0.7380481362342834], [0, 0, 0, 0.1701306104660034, 0, 0, 0], [0, 0, 0, 0, 0.970702052116394, 0, 0.0840241238474845], [0, 0.2939529170592626, 0.9781488180160522, 0, 0.8384366631507874, 0, 0], [0, 0.3768292069435119, 0.8774186968803406, 0.467398464679718, 0, 0.9011183381080629, 0], [0, 0.7990977168083191, 0, 0.5446871519088745, 0.5082175191491842, 0, 0], [0, 0.1095320880413055, 0, 0, 0, 0, 0], [0, 0.3572186181942622, 0, 0, 0, 0, 0], [0.9888333082199096, 0, 0.7707316875457764, 0, 0, 0, 0], [0, 0, 0, 0.9135978519916534, 0, 0, 0], [0.9999566078186036, 0, 0, 0.7479395270347595, 0, 0, 0], [0, 0.4713231921195984, 0, 0.7501673102378845, 0, 0, 0], [0, 0.8097033898035685, 0, 0, 0.7898019552230835, 0, 0], [0.7853492498397827, 0.2264062613248825, 0.6652280688285828, 0.2589850127696991, 0, 0, 0], [0, 0.1148651763796806, 0, 0, 0, 0.9234976172447203, 0], [0, 0.347608208656311, 0, 0.30149710923433304, 0, 0, 0], [0, 0.3033813834190368, 0, 0, 0, 0, 0], [0.9251611828804016, 0, 0.548084999124209, 0.1676252335309982, 0.941423237323761, 0, 0], [0, 0.2315873503684997, 0, 0.0809566527605056, 0.524580717086792, 0, 0], [0, 0.3360502123832702, 0.9642099142074584, 0.6644876599311829, 0, 0, 0], [0.9838617444038392, 0.5211476534605026, 0, 0, 0, 0, 0], [0.7239340543746948, 0.45574740444620443, 0.692572757601738, 0, 0, 0, 0], [0, 0, 0.9572142163912455, 0.523177981376648, 0.386613667011261, 0, 0], [0, 0.9896531701087952, 0.9529829025268556, 0, 0, 0, 0], [0.1780087649822235, 0.5489380608002344, 0.0674691498279571, 0, 0, 0, 0], [0.8553030490875244, 0.8149787485599518, 0.7120435237884521, 0.9060190320014954, 0, 0, 0], [0, 0, 0, 0.3105745315551758, 0, 0, 0], [0.8385533690452576, 0.6635442078113556, 0, 0, 0.695237934589386, 0, 0], [0, 0.5707869082689285, 0, 0, 0, 0, 0], [0, 0.48639462391535443, 0, 0, 0.2835976481437683, 0, 0], [0, 0, 0, 0, 0, 0, 0.089213527739048], [0, 0.9669797420501708, 0, 0.1481857299804687, 0, 0, 0], [0, 0, 0, 0.0787472650408744, 0, 0, 0], [0.9830586910247804, 0.5943448692560196, 0.9935729503631592, 0, 0, 0, 0], [0, 0.584276556968689, 0.3518165349960327, 0.9768030643463136, 0, 0, 0], [0, 0.8027613162994385, 0, 0.4388434439897537, 0, 0, 0], [0, 0.2356525212526321, 0.0331190750002861, 0.0952656045556068, 0, 0, 0.7619056105613708], [0, 0, 0, 0.1374349892139434, 0, 0, 0], [0, 0.1481355801224708, 0, 0, 0, 0, 0], [0, 0.781432569026947, 0, 0.8084046244621277, 0, 0, 0], [0, 0.7181117534637451, 0, 0, 0, 0, 0], [0.5727640390396118, 0.7277653217315674, 0.4432311058044433, 0, 0, 0, 0], [0, 0, 0.3616742491722107, 0, 0.3747145235538482, 0.2872264981269836, 0], [0, 0.3782610297203064, 0, 0.854012668132782, 0.4967236816883087, 0, 0], [0, 0.19986538589000702, 0, 0.3909452259540558, 0, 0.1013399288058281, 0], [0.9928357005119324, 0, 0, 0.7842077016830444, 0.942022740840912, 0, 0], [0, 0.1431154645979404, 0, 0.26619162162144977, 0, 0, 0], [0, 0.5705551132559776, 0, 0, 0, 0, 0], [0, 0.5234284400939941, 0, 0, 0, 0, 0], [0.9374096393585204, 0, 0.8265523314476013, 0, 0, 0, 0], [0.8333739638328552, 0, 0, 0.0599147714674472, 0, 0, 0], [0.9971029162406922, 0, 0, 0.9498622218767802, 0, 0, 0], [0, 0, 0, 0.6124492287635803, 0, 0, 0], [0, 0, 0, 0.1473552584648132, 0.8214776515960693, 0, 0], [0, 0.2070727404206991, 0.13406430929899216, 0, 0, 0, 0], [0, 0.9881119132041931, 0, 0.6079228023687998, 0, 0, 0], [0, 0.3620156894127528, 0.8291893005371094, 0, 0.30824635922908783, 0, 0], [0, 0.5328339437643687, 0.5795112699270248, 0, 0.6723833680152893, 0, 0], [0, 0.5736834853887558, 0, 0, 0, 0, 0], [0, 0, 0.9607513248920441, 0, 0.7815884947776794, 0, 0], [0, 0.2566887885332107, 0.5224303007125854, 0, 0, 0, 0], [0, 0.39935678616166115, 0.9554919600486756, 0.3291648328304291, 0, 0, 0], [0, 0.2196373119950294, 0, 0, 0, 0, 0], [0, 0.46290088444948196, 0, 0, 0, 0, 0.8789078593254089], [0, 0, 0, 0.4082956612110138, 0, 0, 0.5008679628372192], [0.8626933395862579, 0.6772453784942627, 0.8843284845352173, 0.9481538534164428, 0.9478491544723512, 0, 0], [0, 0, 0, 0.662326592952013, 0.9369852542877196, 0, 0], [0.6363880038261414, 0, 0, 0, 0, 0, 0], [0.7153323292732239, 0.6597811728715895, 0, 0, 0, 0, 0], [0, 0.3617240488529205, 0.2684223353862762, 0.8513832688331604, 0, 0, 0], [0, 0.8687215447425842, 0, 0.997450053691864, 0, 0, 0], [0.9999873638153076, 0, 0, 0.9986761212348938, 0.828154444694519, 0, 0], [0.8142076134681702, 0, 0, 0.64506596326828, 0, 0.0192715562880039, 0], [0, 0, 0.7062419652938843, 0, 0, 0, 0], [0.5043565630912781, 0, 0, 0, 0.8240281343460083, 0, 0], [0.4132416844367981, 0, 0, 0, 0, 0, 0], [0, 0, 0.202496200799942, 0.2565383613109588, 0, 0.0394061766564846, 0], [0, 0, 0, 0.7405649721622467, 0, 0, 0], [0, 0.5842433571815491, 0, 0, 0, 0, 0], [0, 0.9134953618049622, 0, 0, 0, 0, 0], [0, 0.6991828382015228, 0, 0.1673416048288345, 0, 0, 0.3717274069786072], [0, 0, 0.0255616270005702, 0, 0, 0, 0], [0, 0.28993574902415276, 0, 0.4029674828052521, 0, 0, 0], [0.3208548724651336, 0, 0.1485220789909362, 0.5123230367898941, 0, 0.4750406146049499, 0], [0, 0.6834388971328735, 0.1843864470720291, 0.36344597488641733, 0, 0, 0], [0, 0.5439689755439758, 0, 0, 0.8017814755439758, 0, 0], [0, 0.7887302041053772, 0, 0.6101733818650246, 0.9875646233558656, 0, 0], [0.1813472509384155, 0.4900151789188385, 0, 0, 0.8510906100273132, 0.2888577878475189, 0], [0, 0.3665411472320556, 0, 0, 0.3824392855167389, 0, 0], [0.804774820804596, 0.2093791365623474, 0, 0.4012947976589203, 0, 0, 0], [0, 0, 0.5327671766281128, 0.1809339076280594, 0.7181224822998047, 0.150772538036108, 0], [0, 0, 0, 0.6814729770024618, 0, 0, 0], [0.44602896459400654, 0.8979423642158508, 0.0385872460901737, 0, 0, 0, 0], [0.8537272810935974, 0.5059460904449224, 0, 0.9962869882583618, 0.5474837422370911, 0, 0], [0, 0, 0.9664808511734008, 0, 0.9849373698234558, 0, 0], [0, 0.5215648636221885, 0, 0.1134360954165458, 0, 0, 0.1888028234243393], [0, 0.6771321818232536, 0.6375916004180908, 0, 0, 0.4533230066299438, 0], [0, 0, 0, 0.31572508811950684, 0, 0, 0], [0, 0.5892202705144882, 0, 0, 0, 0, 0], [0, 0.715386688709259, 0, 0, 0, 0, 0], [0, 0.995960533618927, 0.5360426306724548, 0, 0, 0, 0], [0, 0.736011803150177, 0.791957676410675, 0.6457101404666901, 0, 0, 0], [0.918582558631897, 0, 0.6923561692237854, 0, 0.6384051144123077, 0, 0], [0.7924186786015829, 0.46576949767768383, 0, 0.1192408576607704, 0, 0, 0], [0, 0, 0, 0.9482183456420898, 0, 0, 0], [0, 0, 0, 0.0853188708424568, 0, 0, 0], [0, 0, 0.42654329538345337, 0, 0.198809415102005, 0, 0], [0, 0.296381413936615, 0, 0, 0, 0, 0], [0, 0.8001119196414948, 0, 0.7656692663828532, 0, 0, 0], [0, 0, 0, 0.9085761904716492, 0, 0, 0], [0.5727640390396118, 0.5180519148707389, 0, 0.6036589443683623, 0, 0, 0], [0, 0.2489706575870514, 0, 0, 0, 0, 0], [0.7382850050926208, 0.9706627130508424, 0, 0.1722590327262878, 0, 0, 0], [0, 0, 0.9484108090400696, 0, 0, 0, 0], [0, 0.47814201315244037, 0.3657584190368652, 0, 0, 0.7399091124534607, 0], [0, 0.1228229776024818, 0, 0.6974291205406189, 0, 0, 0], [0.9998968839645386, 0, 0.7807781100273132, 0, 0, 0, 0], [0, 0.34584666043519974, 0.1672731637954712, 0.5999831389635801, 0, 0, 0], [0, 0, 0, 0, 0.1218799129128456, 0, 0], [0, 0.9999085664749146, 0, 0, 0, 0, 0], [0, 0.0870582610368728, 0, 0.3847594857215881, 0, 0, 0], [0, 0.8165372610092163, 0, 0, 0, 0, 0], [0, 0, 0, 0.0598405748605728, 0, 0, 0], [0.8718810081481934, 0, 0, 0, 0, 0, 0], [0.3804613351821899, 0.5263410806655884, 0, 0.7472032010555267, 0, 0, 0], [0, 0, 0, 0.9951828122138976, 0, 0, 0], [0.9918335676193236, 0, 0, 0.5289152264595032, 0, 0, 0], [0.3766694664955139, 0.8329246044158936, 0, 0, 0, 0, 0], [0, 0, 0, 0.2462428659200668, 0, 0, 0], [0, 0.3003977686166763, 0.1341946125030517, 0.3317366242408752, 0.2069655805826187, 0, 0], [0, 0.8575235605239868, 0, 0.8620091080665588, 0, 0, 0], [0, 0.97879159450531, 0, 0.4058115966618061, 0.587875247001648, 0, 0], [0, 0, 0, 0.2798959910869598, 0, 0, 0], [0, 0.5114593505859375, 0, 0.5954097136855125, 0, 0, 0.36953024566173553], [0, 0.971643328666687, 0, 0.9005259275436401, 0.894963264465332, 0, 0], [0, 0.3621469140052795, 0, 0.6304070353507996, 0, 0.1878715008497238, 0], [0.9902136921882628, 0.7737021446228027, 0, 0, 0.8860723376274109, 0, 0], [0, 0.1853985637426376, 0, 0.8094807267189026, 0.5984287261962891, 0, 0], [0, 0.8410638570785522, 0, 0, 0.5742425918579102, 0, 0], [0, 0.39448509365320206, 0, 0, 0.9975784420967102, 0, 0], [0, 0, 0, 0.0379988029599189, 0, 0.2622433602809906, 0], [0, 0, 0.6475211679935455, 0.9948638081550598, 0.8861631155014038, 0, 0], [0, 0.3873155762751897, 0, 0.5641135573387146, 0, 0, 0], [0, 0, 0, 0.30073313042521477, 0.41062226891517634, 0, 0], [0.890795886516571, 0, 0, 0.942528247833252, 0, 0, 0], [0, 0, 0, 0.9442080855369568, 0, 0, 0], [0, 0.4016607999801636, 0, 0, 0, 0, 0], [0, 0.1103745773434639, 0, 0, 0, 0, 0], [0, 0, 0.2744899392127991, 0, 0, 0, 0], [0, 0, 0, 0.9999927282333374, 0.9982655644416808, 0, 0], [0, 0.6390618085861206, 0.1381522268056869, 0.2653797268867492, 0.3480162918567657, 0, 0], [0, 0, 0, 0.41743443533778185, 0, 0, 0], [0.9059877991676332, 0.999525785446167, 0, 0.3291953913867473, 0, 0, 0], [0.846954882144928, 0.9847377836704254, 0, 0.09320053582390143, 0, 0, 0], [0.9663292765617372, 0, 0.26960317417979235, 0.5588350892066956, 0, 0, 0], [0, 0, 0, 0.23991772780815757, 0, 0, 0], [0, 0.2797872424125671, 0, 0, 0, 0, 0], [0, 0.4227174073457718, 0, 0.5048622290293375, 0.9865593910217284, 0, 0], [0.6334412693977356, 0.4946913868188858, 0, 0, 0.753395676612854, 0, 0], [0.942265659570694, 0, 0.1436433941125869, 0, 0, 0, 0], [0, 0, 0, 0.2865998148918152, 0.8375911116600037, 0, 0], [0, 0.0597660616040229, 0, 0.1155664026737213, 0, 0, 0], [0, 0.3219424486160278, 0, 0, 0.8354579210281372, 0, 0], [0, 0.933258593082428, 0.4212210476398468, 0, 0, 0, 0], [0, 0, 0, 0.18798514641821384, 0, 0, 0], [0, 0.5522451996803284, 0, 0.5758950114250183, 0, 0, 0], [0.875389575958252, 0.9254366755485536, 0, 0, 0, 0, 0.6699991822242737], [0, 0.54494721442461, 0, 0.763841837644577, 0.9464755654335022, 0, 0], [0, 0.6046744585037231, 0, 0, 0, 0, 0], [0, 0.0170830581337213, 0.2428068369626999, 0, 0.441216766834259, 0, 0], [0, 0, 0, 0.5669242739677429, 0, 0.0568880029022693, 0], [0, 0.0558788292109966, 0.3875651359558105, 0, 0.2538265436887741, 0, 0], [0, 0, 0, 0.6182939410209656, 0.2835976481437683, 0, 0], [0, 0.0755827650427818, 0, 0.5474700927734375, 0, 0, 0], [0, 0, 0, 0, 0.357115775346756, 0, 0], [0, 0, 0.1239846721291542, 0, 0, 0, 0.2588282823562622], [0, 0.7061436176300049, 0.3597057163715362, 0.1767493486404419, 0.6948080658912659, 0, 0], [0, 0.5949000120162964, 0, 0, 0.367022693157196, 0, 0], [0, 0.1204182207584381, 0.4491068422794342, 0, 0.7429479956626892, 0, 0], [0, 0.1374718397855758, 0.9613185524940492, 0, 0.6473440527915955, 0, 0], [0, 0, 0, 0.8424149751663208, 0.8534520268440247, 0.8857289552688599, 0], [0, 0, 0, 0.7495318949222565, 0, 0, 0], [0.3275781273841858, 0.2558297514915466, 0, 0, 0, 0, 0], [0, 0, 0, 0.4327642172574997, 0.3491645157337188, 0, 0], [0, 0.18942046165466303, 0, 0, 0, 0, 0], [0, 0.93048095703125, 0, 0, 0, 0, 0], [0, 0.45750362426042557, 0.1443936675786972, 0.178844466805458, 0, 0, 0.129309669137001], [0, 0.8548882156610489, 0, 0, 0, 0, 0], [0, 0.6803147246440252, 0.4102682471275329, 0, 0, 0, 0], [0.6531750559806824, 0.5526596307754517, 0, 0, 0.4005244076251983, 0, 0], [0, 0.1245761662721633, 0, 0.13963700830936432, 0, 0, 0], [0, 0, 0, 0.5437676422297955, 0, 0.9987727999687196, 0], [0, 0, 0, 0, 0.9743108749389648, 0, 0], [0, 0.8281846841176351, 0, 0.9999954700469972, 0, 0, 0], [0.9340490698814392, 0.5792956948280334, 0, 0, 0, 0, 0], [0.9999409914016724, 0, 0.7320988178253174, 0.8242762088775635, 0, 0, 0], [0.9984747767448424, 0.9791644811630248, 0, 0, 0, 0, 0], [0, 0.2230485007166862, 0, 0.2112819850444793, 0, 0, 0], [0, 0, 0.8931217789649963, 0, 0.2104215621948242, 0, 0], [0, 0.6784105896949768, 0, 0.6258914073308309, 0, 0, 0], [0.9975273609161377, 0, 0, 0.5186004638671875, 0, 0, 0], [0.581733763217926, 0, 0, 0.8559986352920532, 0, 0, 0], [0.3653983771800995, 0.9760369658470154, 0, 0, 0, 0, 0], [0, 0, 0.4707983136177063, 0, 0.3716106712818146, 0, 0], [0, 0.5499006360769272, 0.1369954645633697, 0.3679050207138061, 0, 0.8253938555717468, 0], [0, 0.9348942637443542, 0, 0, 0, 0, 0], [0.8243325352668762, 0, 0, 0.9895742535591124, 0, 0, 0], [0, 0.9211660623550415, 0.7448468804359436, 0.8644008040428162, 0, 0, 0], [0, 0.6340291500091553, 0, 0.1172443553805351, 0, 0, 0], ...]
distance_matrix = cdist(workers_df['data'].tolist(), workers_df['data'].tolist(), 'euclidean')
distance_df = pd.DataFrame(distance_matrix, index=workers_df.wid, columns=workers_df.wid)
# g = sns.clustermap(distance_df, center=0, cmap="vlag",
# dendrogram_ratio=(.1, .2),
# cbar_pos=(.02, .32, .03, .2),
# linewidths=.75, figsize=(12, 13))
len(distance_df)
4644
distance_block_df = distance_df.iloc[:100,:100]
g = sns.clustermap(distance_block_df, center=0, cmap="vlag",
dendrogram_ratio=(.1, .2),
cbar_pos=(.02, .32, .03, .2),
linewidths=.75, figsize=(12, 13))
/Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance. warnings.warn(msg) /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/seaborn/matrix.py:530: ClusterWarning: scipy.cluster: The symmetric non-negative hollow observation matrix looks suspiciously like an uncondensed distance matrix linkage = hierarchy.linkage(self.array, method=self.method, /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance. warnings.warn(msg) /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/seaborn/matrix.py:530: ClusterWarning: scipy.cluster: The symmetric non-negative hollow observation matrix looks suspiciously like an uncondensed distance matrix linkage = hierarchy.linkage(self.array, method=self.method,
len(distance_df.columns)
4644
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/happy_third.jpg"
Image(filename=image_path, width=600, height=600)
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/workflow.jpg"
Image(filename=image_path, width=600, height=600)
👷In the third part, progress is being continued. The final version will finished 1-2 days after
🌟The current idea is to use the previous word cloud to infer the scenario, because the third part is mainly to explore "how", so for example, it is explored that for married people, a possible way to improve happiness is for couples to go home together after get off work. Parents pick up and drop off children after school, and single people may get orders from big customers, promotions and salary increases, social recognition, etc. through the workplace.
📒The picture above is the brainstorming flow chart for the third part
## Still Working ...
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
cleaned_hm_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/cleaned_hm.csv"
cleaned_hm_df = pd.read_csv(cleaned_hm_csv_file)
demographic_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv"
demographic_df = pd.read_csv(demographic_csv_file)
cleaned_hm_df = cleaned_hm_df.dropna(subset=['ground_truth_category'])
cleaned_hm_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|
| 3 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding |
| 5 | 27678 | 45 | 24h | I meditated last night. | I meditated last night. | True | 1 | leisure | leisure |
| 24 | 27697 | 498 | 24h | My grandmother start to walk from the bed afte... | My grandmother start to walk from the bed afte... | True | 1 | affection | affection |
| 32 | 27705 | 5732 | 24h | I picked my daughter up from the airport and w... | I picked my daughter up from the airport and w... | True | 1 | bonding | affection |
| 42 | 27715 | 2272 | 24h | when i received flowers from my best friend | when i received flowers from my best friend | True | 1 | bonding | bonding |
cleaned_hm_df_merge = cleaned_hm_df.merge(demographic_df,on='wid')
cleaned_hm_df_merge.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding | 28 | DNK | f | married | n |
| 1 | 49534 | 206 | 24h | I bought my wedding shoes! | I bought my wedding shoes! | True | 1 | achievement | achievement | 28 | DNK | f | married | n |
| 2 | 52451 | 206 | 24h | My new watch arrived in the mail! | My new watch arrived in the mail! | True | 1 | achievement | achievement | 28 | DNK | f | married | n |
| 3 | 84125 | 206 | 3m | We watched Rocky Horror Picture Show for the f... | We watched Rocky Horror Picture Show for the f... | True | 1 | leisure | leisure | 28 | DNK | f | married | n |
| 4 | 85125 | 206 | 3m | We played Mario Kart while drunk. | We played Mario Kart while drunk. | True | 1 | leisure | leisure | 28 | DNK | f | married | n |
cleaned_hm_adult_cleaned_df = cleaned_hm_df_merge[pd.to_numeric(cleaned_hm_df_merge['age'],errors='coerce').notnull()]
cleaned_hm_adult_cleaned_df['age'] = cleaned_hm_adult_cleaned_df['age'].astype(str).astype(float).astype(int)
cleaned_hm_adult_cleaned_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['age']>30]
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/4083055248.py:2: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy cleaned_hm_adult_cleaned_df['age'] = cleaned_hm_adult_cleaned_df['age'].astype(str).astype(float).astype(int)
cleaned_hm_adult_cleaned_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 16 | 27705 | 5732 | 24h | I picked my daughter up from the airport and w... | I picked my daughter up from the airport and w... | True | 1 | bonding | affection | 64 | USA | f | divorced | y |
| 20 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 61 | USA | f | married | y |
| 21 | 27923 | 389 | 24h | My daughter visited me at evening | My daughter visited me at evening | True | 1 | affection | affection | 61 | USA | f | married | y |
| 22 | 45297 | 389 | 24h | I meet my son and ate a lunch with him | I meet my son and ate a lunch with him | True | 1 | affection | affection | 61 | USA | f | married | y |
| 23 | 83193 | 389 | 3m | I meet my daughter special one | I meet my daughter special one | True | 1 | affection | affection | 61 | USA | f | married | y |
adult_married_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital']=="married"]
adult_married_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 20 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 61 | USA | f | married | y |
| 21 | 27923 | 389 | 24h | My daughter visited me at evening | My daughter visited me at evening | True | 1 | affection | affection | 61 | USA | f | married | y |
| 22 | 45297 | 389 | 24h | I meet my son and ate a lunch with him | I meet my son and ate a lunch with him | True | 1 | affection | affection | 61 | USA | f | married | y |
| 23 | 83193 | 389 | 3m | I meet my daughter special one | I meet my daughter special one | True | 1 | affection | affection | 61 | USA | f | married | y |
| 24 | 89903 | 389 | 3m | I meet my son girlfriend | I meet my son girlfriend | True | 1 | affection | affection | 61 | USA | f | married | y |
adult_married_df.groupby(['ground_truth_category']).agg(['count'])
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | count | count | count | count | count | count | count | count | count | count | count | count | |
| ground_truth_category | |||||||||||||
| achievement | 1028 | 1028 | 1028 | 1028 | 1028 | 1028 | 1028 | 1028 | 1028 | 1027 | 1027 | 1028 | 1028 |
| affection | 1706 | 1706 | 1706 | 1706 | 1706 | 1706 | 1706 | 1706 | 1706 | 1704 | 1704 | 1706 | 1704 |
| bonding | 342 | 342 | 342 | 342 | 342 | 342 | 342 | 342 | 342 | 342 | 342 | 342 | 342 |
| enjoy_the_moment | 323 | 323 | 323 | 323 | 323 | 323 | 323 | 323 | 323 | 322 | 323 | 323 | 323 |
| exercise | 37 | 37 | 37 | 37 | 37 | 37 | 37 | 37 | 37 | 36 | 37 | 37 | 37 |
| leisure | 227 | 227 | 227 | 227 | 227 | 227 | 227 | 227 | 227 | 227 | 227 | 227 | 227 |
| nature | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 |
marital_recs = []
sns.set()
for martial_state in cleaned_hm_adult_cleaned_df['marital'].unique():
rec = {
"name":martial_state,
"count":len(cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital'] == martial_state])
}
marital_recs.append(rec)
marital_state_df = pd.DataFrame(marital_recs)
sns.barplot(data=marital_state_df,x="name",y="count")
<Axes: xlabel='name', ylabel='count'>
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_adult_cleaned_df['parenthood'].unique():
rec = {
"name":parenthood_state,
"count":len(cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['parenthood'] == parenthood_state])
}
recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
<Axes: xlabel='name', ylabel='count'>
cleaned_hm_adult_married_cleaned_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital']=='married']
cleaned_hm_adult_married_cleaned_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 20 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 61 | USA | f | married | y |
| 21 | 27923 | 389 | 24h | My daughter visited me at evening | My daughter visited me at evening | True | 1 | affection | affection | 61 | USA | f | married | y |
| 22 | 45297 | 389 | 24h | I meet my son and ate a lunch with him | I meet my son and ate a lunch with him | True | 1 | affection | affection | 61 | USA | f | married | y |
| 23 | 83193 | 389 | 3m | I meet my daughter special one | I meet my daughter special one | True | 1 | affection | affection | 61 | USA | f | married | y |
| 24 | 89903 | 389 | 3m | I meet my son girlfriend | I meet my son girlfriend | True | 1 | affection | affection | 61 | USA | f | married | y |
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_adult_married_cleaned_df['parenthood'].unique():
rec = {
"name":parenthood_state,
"count":len(cleaned_hm_adult_married_cleaned_df[cleaned_hm_adult_married_cleaned_df['parenthood'] == parenthood_state])
}
recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
<Axes: xlabel='name', ylabel='count'>
cleaned_hm_adult_married_cleaned_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 20 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 61 | USA | f | married | y |
| 21 | 27923 | 389 | 24h | My daughter visited me at evening | My daughter visited me at evening | True | 1 | affection | affection | 61 | USA | f | married | y |
| 22 | 45297 | 389 | 24h | I meet my son and ate a lunch with him | I meet my son and ate a lunch with him | True | 1 | affection | affection | 61 | USA | f | married | y |
| 23 | 83193 | 389 | 3m | I meet my daughter special one | I meet my daughter special one | True | 1 | affection | affection | 61 | USA | f | married | y |
| 24 | 89903 | 389 | 3m | I meet my son girlfriend | I meet my son girlfriend | True | 1 | affection | affection | 61 | USA | f | married | y |
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
cleaned_hm_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/cleaned_hm.csv"
cleaned_hm_df = pd.read_csv(cleaned_hm_csv_file)
demographic_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv"
demographic_df = pd.read_csv(demographic_csv_file)
cleaned_hm_df = cleaned_hm_df.dropna(subset=['ground_truth_category'])
len(demographic_df)
10844
len(cleaned_hm_df)
14125
cleaned_hm_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|
| 3 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding |
| 5 | 27678 | 45 | 24h | I meditated last night. | I meditated last night. | True | 1 | leisure | leisure |
| 24 | 27697 | 498 | 24h | My grandmother start to walk from the bed afte... | My grandmother start to walk from the bed afte... | True | 1 | affection | affection |
| 32 | 27705 | 5732 | 24h | I picked my daughter up from the airport and w... | I picked my daughter up from the airport and w... | True | 1 | bonding | affection |
| 42 | 27715 | 2272 | 24h | when i received flowers from my best friend | when i received flowers from my best friend | True | 1 | bonding | bonding |
cleaned_hm_df_merge = cleaned_hm_df.merge(demographic_df,on='wid')
len(cleaned_hm_df_merge)
14125
cleaned_hm_df_merge.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding | 28 | DNK | f | married | n |
| 1 | 49534 | 206 | 24h | I bought my wedding shoes! | I bought my wedding shoes! | True | 1 | achievement | achievement | 28 | DNK | f | married | n |
| 2 | 52451 | 206 | 24h | My new watch arrived in the mail! | My new watch arrived in the mail! | True | 1 | achievement | achievement | 28 | DNK | f | married | n |
| 3 | 84125 | 206 | 3m | We watched Rocky Horror Picture Show for the f... | We watched Rocky Horror Picture Show for the f... | True | 1 | leisure | leisure | 28 | DNK | f | married | n |
| 4 | 85125 | 206 | 3m | We played Mario Kart while drunk. | We played Mario Kart while drunk. | True | 1 | leisure | leisure | 28 | DNK | f | married | n |
cleaned_hm_df_merge['marital'].unique()
array(['married', 'single', 'divorced', 'separated', 'widowed', nan],
dtype=object)
cleaned_hm_df_merge['marital'].isna().sum()
22
# Initialize an empty list to store records of counts for each marital state
marital_recs = []
# Set the aesthetic style of the plots
sns.set()
# Loop over each unique marital state in the merged dataframe containing cleaned happiness messages
for marital_state in cleaned_hm_df_merge['marital'].unique():
# Count the number of occurrences for the current marital state
count = cleaned_hm_df_merge[cleaned_hm_df_merge['marital'] == marital_state].shape[0]
# Create a dictionary record with the marital state name and its count
rec = {"name": marital_state, "count": count}
# Append the record to the list of marital records
marital_recs.append(rec)
# Convert the list of records into a pandas DataFrame
marital_state_df = pd.DataFrame(marital_recs)
# Use Seaborn to create a bar plot visualizing the frequency of each marital state
sns.barplot(data=marital_state_df, x="name", y="count")
<Axes: xlabel='name', ylabel='count'>
marital_state_df
| name | count | |
|---|---|---|
| 0 | married | 6059 |
| 1 | single | 7431 |
| 2 | divorced | 471 |
| 3 | separated | 84 |
| 4 | widowed | 58 |
| 5 | NaN | 0 |
marital_state_df['count'].sum()
14103
cleaned_hm_df_merge['parenthood'].unique()
array(['n', 'y', nan], dtype=object)
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_df_merge['parenthood'].unique():
rec = {
"name":parenthood_state,
"count":len(cleaned_hm_df_merge[cleaned_hm_df_merge['parenthood'] == parenthood_state])
}
recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
<Axes: xlabel='name', ylabel='count'>
cleaned_hm_adult_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding | 28 | DNK | f | married | n |
| 1 | 49534 | 206 | 24h | I bought my wedding shoes! | I bought my wedding shoes! | True | 1 | achievement | achievement | 28 | DNK | f | married | n |
| 2 | 52451 | 206 | 24h | My new watch arrived in the mail! | My new watch arrived in the mail! | True | 1 | achievement | achievement | 28 | DNK | f | married | n |
| 3 | 84125 | 206 | 3m | We watched Rocky Horror Picture Show for the f... | We watched Rocky Horror Picture Show for the f... | True | 1 | leisure | leisure | 28 | DNK | f | married | n |
| 4 | 85125 | 206 | 3m | We played Mario Kart while drunk. | We played Mario Kart while drunk. | True | 1 | leisure | leisure | 28 | DNK | f | married | n |
cleaned_hm_adult_df = cleaned_hm_df_merge.dropna(subset=['age'])
# This will print any value that cannot be converted to a float
for value in cleaned_hm_df_merge['age']:
try:
float(value)
except ValueError:
print(value)
prefer not to say prefer not to say prefer not to say prefer not to say prefer not to say prefer not to say prefer not to say prefer not to say prefer not to say prefer not to say prefer not to say prefer not to say čá
# Replace non-numeric values with NaN
cleaned_hm_df_merge['age'] = pd.to_numeric(cleaned_hm_df_merge['age'], errors='coerce')
# Now you can drop NaN values if that's the chosen approach
cleaned_hm_df_merge = cleaned_hm_df_merge.dropna(subset=['age'])
# Perform the filtering
cleaned_hm_adult_df = cleaned_hm_df_merge[cleaned_hm_df_merge['age'] > 30]
cleaned_hm_adult_cleaned_df = cleaned_hm_adult_df[pd.to_numeric(cleaned_hm_adult_df['age'],errors='coerce').notnull()]
cleaned_hm_adult_cleaned_df['age'] = cleaned_hm_adult_cleaned_df['age'].astype(str).astype(float).astype(int)
cleaned_hm_adult_cleaned_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 16 | 27705 | 5732 | 24h | I picked my daughter up from the airport and w... | I picked my daughter up from the airport and w... | True | 1 | bonding | affection | 64 | USA | f | divorced | y |
| 20 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 61 | USA | f | married | y |
| 21 | 27923 | 389 | 24h | My daughter visited me at evening | My daughter visited me at evening | True | 1 | affection | affection | 61 | USA | f | married | y |
| 22 | 45297 | 389 | 24h | I meet my son and ate a lunch with him | I meet my son and ate a lunch with him | True | 1 | affection | affection | 61 | USA | f | married | y |
| 23 | 83193 | 389 | 3m | I meet my daughter special one | I meet my daughter special one | True | 1 | affection | affection | 61 | USA | f | married | y |
cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['age']>30]
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 16 | 27705 | 5732 | 24h | I picked my daughter up from the airport and w... | I picked my daughter up from the airport and w... | True | 1 | bonding | affection | 64 | USA | f | divorced | y |
| 20 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 61 | USA | f | married | y |
| 21 | 27923 | 389 | 24h | My daughter visited me at evening | My daughter visited me at evening | True | 1 | affection | affection | 61 | USA | f | married | y |
| 22 | 45297 | 389 | 24h | I meet my son and ate a lunch with him | I meet my son and ate a lunch with him | True | 1 | affection | affection | 61 | USA | f | married | y |
| 23 | 83193 | 389 | 3m | I meet my daughter special one | I meet my daughter special one | True | 1 | affection | affection | 61 | USA | f | married | y |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 14116 | 128565 | 5216 | 24h | I saw a romantic comedy. | I saw a romantic comedy. | True | 1 | leisure | leisure | 40 | USA | m | single | y |
| 14118 | 128597 | 5826 | 24h | I found a little extra money in one of my onli... | I found a little extra money in one of my onli... | True | 1 | achievement | achievement | 44 | USA | m | single | n |
| 14119 | 128609 | 6437 | 24h | After diner I had a nice family walk with my c... | After diner I had a nice family walk with my c... | True | 1 | affection | affection | 31 | LVA | m | married | y |
| 14121 | 128631 | 8937 | 24h | My best out of town friend called to announce ... | My best out of town friend called to announce ... | True | 1 | bonding | bonding | 61 | USA | m | single | n |
| 14123 | 128758 | 4428 | 24h | Yesterday my relations came to my house. That ... | Yesterday my relations came to my house. That ... | True | 2 | affection | affection | 34 | IND | f | married | y |
6237 rows × 14 columns
len(cleaned_hm_adult_cleaned_df)
6237
cleaned_hm_adult_cleaned_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['age']>30]
len(cleaned_hm_adult_cleaned_df)
6237
marital_recs = []
sns.set()
for martial_state in cleaned_hm_adult_cleaned_df['marital'].unique():
rec = {
"name":martial_state,
"count":len(cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital'] == martial_state])
}
marital_recs.append(rec)
marital_state_df = pd.DataFrame(marital_recs)
sns.barplot(data=marital_state_df,x="name",y="count")
<Axes: xlabel='name', ylabel='count'>
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_adult_cleaned_df['parenthood'].unique():
rec = {
"name":parenthood_state,
"count":len(cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['parenthood'] == parenthood_state])
}
recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
<Axes: xlabel='name', ylabel='count'>
cleaned_hm_adult_married_cleaned_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital']=='married']
cleaned_hm_adult_married_cleaned_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 20 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 61 | USA | f | married | y |
| 21 | 27923 | 389 | 24h | My daughter visited me at evening | My daughter visited me at evening | True | 1 | affection | affection | 61 | USA | f | married | y |
| 22 | 45297 | 389 | 24h | I meet my son and ate a lunch with him | I meet my son and ate a lunch with him | True | 1 | affection | affection | 61 | USA | f | married | y |
| 23 | 83193 | 389 | 3m | I meet my daughter special one | I meet my daughter special one | True | 1 | affection | affection | 61 | USA | f | married | y |
| 24 | 89903 | 389 | 3m | I meet my son girlfriend | I meet my son girlfriend | True | 1 | affection | affection | 61 | USA | f | married | y |
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_adult_married_cleaned_df['parenthood'].unique():
rec = {
"name":parenthood_state,
"count":len(cleaned_hm_adult_married_cleaned_df[cleaned_hm_adult_married_cleaned_df['parenthood'] == parenthood_state])
}
recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
<Axes: xlabel='name', ylabel='count'>
cleaned_hm_adult_married_cleaned_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 20 | 27723 | 389 | 24h | I went shopping | I went shopping | True | 1 | leisure | leisure | 61 | USA | f | married | y |
| 21 | 27923 | 389 | 24h | My daughter visited me at evening | My daughter visited me at evening | True | 1 | affection | affection | 61 | USA | f | married | y |
| 22 | 45297 | 389 | 24h | I meet my son and ate a lunch with him | I meet my son and ate a lunch with him | True | 1 | affection | affection | 61 | USA | f | married | y |
| 23 | 83193 | 389 | 3m | I meet my daughter special one | I meet my daughter special one | True | 1 | affection | affection | 61 | USA | f | married | y |
| 24 | 89903 | 389 | 3m | I meet my son girlfriend | I meet my son girlfriend | True | 1 | affection | affection | 61 | USA | f | married | y |
import torch
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
chm_csv_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/cleaned_hm.csv"
demographic_csv_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv"
chm_df = pd.read_csv(chm_csv_path)
chm_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 27673 | 2053 | 24h | I went on a successful date with someone I fel... | I went on a successful date with someone I fel... | True | 1 | NaN | affection |
| 1 | 27674 | 2 | 24h | I was happy when my son got 90% marks in his e... | I was happy when my son got 90% marks in his e... | True | 1 | NaN | affection |
| 2 | 27675 | 1936 | 24h | I went to the gym this morning and did yoga. | I went to the gym this morning and did yoga. | True | 1 | NaN | exercise |
| 3 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding |
| 4 | 27677 | 6227 | 24h | I went with grandchildren to butterfly display... | I went with grandchildren to butterfly display... | True | 1 | NaN | affection |
demographic_df = pd.read_csv(demographic_csv_path)
demographic_df.head()
| wid | age | country | gender | marital | parenthood | |
|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y |
| 1 | 2 | 29.0 | IND | m | married | y |
| 2 | 3 | 25 | IND | m | single | n |
| 3 | 4 | 32 | USA | m | married | y |
| 4 | 5 | 29 | USA | m | married | y |
merge_df = demographic_df.merge(chm_df,on='wid')
merge_df.head()
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y | 27854 | 24h | My wife and i are celebrating our ten year ann... | My wife and i are celebrating our ten year ann... | True | 1 | NaN | affection |
| 1 | 1 | 37.0 | USA | m | married | y | 28054 | 24h | My mother called out of the blue to tell me ho... | My mother called out of the blue to tell me ho... | True | 1 | affection | affection |
| 2 | 1 | 37.0 | USA | m | married | y | 28254 | 24h | Today i took the day off from my part time job... | Today i took the day off from my part time job... | True | 1 | NaN | affection |
| 3 | 1 | 37.0 | USA | m | married | y | 28489 | 24h | I just got a $21 bonus on a mturk task! | I just got a $21 bonus on a mturk task! | True | 1 | NaN | achievement |
| 4 | 1 | 37.0 | USA | m | married | y | 28991 | 24h | My wife cooked me a surprise dinner to take to... | My wife cooked me a surprise dinner to take to... | True | 1 | NaN | affection |
merge_df.drop_duplicates(subset=['wid'],inplace=True)
merge_df.head()
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y | 27854 | 24h | My wife and i are celebrating our ten year ann... | My wife and i are celebrating our ten year ann... | True | 1 | NaN | affection |
| 90 | 2 | 29.0 | IND | m | married | y | 27674 | 24h | I was happy when my son got 90% marks in his e... | I was happy when my son got 90% marks in his e... | True | 1 | NaN | affection |
| 174 | 3 | 25 | IND | m | single | n | 27681 | 24h | YESTERDAY MY MOMS BIRTHDAY SO I ENJOYED | YESTERDAY MY MOMS BIRTHDAY SO I ENJOYED | True | 1 | NaN | enjoy_the_moment |
| 258 | 4 | 32 | USA | m | married | y | 27714 | 24h | I received an award for my work on a tough loa... | I received an award for my work on a tough loa... | True | 1 | NaN | achievement |
| 354 | 5 | 29 | USA | m | married | y | 27838 | 24h | I was able to play my video game that I enjoy ... | I was able to play my video game that I enjoy ... | True | 1 | leisure | leisure |
merge_df = merge_df[['wid','age','country','gender','marital','parenthood','ground_truth_category','predicted_category']]
merge_df.head()
| wid | age | country | gender | marital | parenthood | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y | NaN | affection |
| 90 | 2 | 29.0 | IND | m | married | y | NaN | affection |
| 174 | 3 | 25 | IND | m | single | n | NaN | enjoy_the_moment |
| 258 | 4 | 32 | USA | m | married | y | NaN | achievement |
| 354 | 5 | 29 | USA | m | married | y | leisure | leisure |
merge_df['country'].unique()
array(['USA', 'IND', 'VNM', 'THA', 'GBR', 'SRB', 'BGR', 'BRA', 'BGD',
'CAN', 'PRT', 'PHL', 'VEN', 'MKD', 'DOM', 'DNK', 'IDN', 'AUS',
'URY', 'JAM', 'EGY', 'DEU', 'KEN', 'MDA', 'IRL', 'NGA', 'ALB',
'MEX', 'TTO', 'KWT', 'RUS', 'LTU', 'PRI', 'SWE', 'PER', 'PAK',
'TUR', 'FRA', 'ARE', 'COL', 'GRC', 'ROU', 'ARM', 'MAC', 'SGP',
'UGA', 'POL', 'NIC', 'JPN', 'ESP', 'TWN', 'FIN', 'ITA', 'GHA',
'AUT', 'NPL', 'NZL', 'NLD', 'KOR', 'LKA', 'KNA', 'ZAF', 'BEL',
'ISL', 'DZA', 'AFG', 'GMB', nan, 'ASM', 'MYS', 'CZE', 'MLT', 'SVN',
'UMI', 'ARG', 'SAU', 'EST', 'ECU', 'ZMB', 'CHL', 'MAR', 'ISR',
'LVA', 'KAZ', 'NOR', 'CRI', 'BRB', 'HRV', 'ETH', 'TCA', 'BHS',
'VIR', 'GTM', 'IRQ', 'HKG', 'MUS', 'TUN', 'SUR', 'SLV', 'CYP',
'UKR'], dtype=object)
country_series = merge_df['country'].value_counts()[:10]
# country_series
sns.set()
barplot = sns.barplot(x=country_series.index,y=country_series.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
merge_country_df = merge_df[merge_df['country'] == 'USA']
merge_country_df.head()
| wid | age | country | gender | marital | parenthood | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y | NaN | affection |
| 258 | 4 | 32 | USA | m | married | y | NaN | achievement |
| 354 | 5 | 29 | USA | m | married | y | leisure | leisure |
| 510 | 7 | 34 | USA | m | married | y | achievement | achievement |
| 678 | 9 | 61 | USA | f | married | y | NaN | affection |
merge_country_df['age'] = pd.to_numeric(merge_country_df['age'], errors='coerce')
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/1739328044.py:1: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy merge_country_df['age'] = pd.to_numeric(merge_country_df['age'], errors='coerce')
merge_country_df['age'] = merge_country_df['age'].fillna(0).astype(int)
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/813599853.py:1: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy merge_country_df['age'] = merge_country_df['age'].fillna(0).astype(int)
marital_series = merge_country_df[merge_country_df['age']>16]['marital'].value_counts()
barplot = sns.barplot(x=marital_series.index,y=marital_series.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
age_gt_16_df = merge_country_df[merge_country_df['age']>16]
age_gt_16_df.head()
| wid | age | country | gender | marital | parenthood | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37 | USA | m | married | y | NaN | affection |
| 258 | 4 | 32 | USA | m | married | y | NaN | achievement |
| 354 | 5 | 29 | USA | m | married | y | leisure | leisure |
| 510 | 7 | 34 | USA | m | married | y | achievement | achievement |
| 678 | 9 | 61 | USA | f | married | y | NaN | affection |
bins = [16, 20, 25, 30,35,40,45,50,55,60]
labels = ['16-20', '21-25', '26-30','31-35', '36-40', '41-45','46-50', '51-55', '56-60']
age_gt_16_df['range'] = pd.cut(age_gt_16_df['age'], bins=bins, labels=labels)
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/2921349568.py:3: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy age_gt_16_df['range'] = pd.cut(age_gt_16_df['age'], bins=bins, labels=labels)
age_gt_16_df.head()
| wid | age | country | gender | marital | parenthood | ground_truth_category | predicted_category | range | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37 | USA | m | married | y | NaN | affection | 36-40 |
| 258 | 4 | 32 | USA | m | married | y | NaN | achievement | 31-35 |
| 354 | 5 | 29 | USA | m | married | y | leisure | leisure | 26-30 |
| 510 | 7 | 34 | USA | m | married | y | achievement | achievement | 31-35 |
| 678 | 9 | 61 | USA | f | married | y | NaN | affection | NaN |
range_count = age_gt_16_df['range'].value_counts()
barplot = sns.barplot(x=range_count.index,y=range_count.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
married_range_count = age_gt_16_df[age_gt_16_df['marital']=='married']['range'].value_counts()
barplot = sns.barplot(x=married_range_count.index,y=married_range_count.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
sinle_range_count = age_gt_16_df[age_gt_16_df['marital']=='single']['range'].value_counts()
barplot = sns.barplot(x=sinle_range_count.index,y=sinle_range_count.values)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
single_df = age_gt_16_df[age_gt_16_df['marital']=='single']
single_df
| wid | age | country | gender | marital | parenthood | ground_truth_category | predicted_category | range | |
|---|---|---|---|---|---|---|---|---|---|
| 765 | 10 | 27 | USA | m | single | n | NaN | exercise | 26-30 |
| 936 | 12 | 25 | USA | f | single | n | NaN | affection | 21-25 |
| 1239 | 16 | 35 | USA | m | single | n | NaN | affection | 31-35 |
| 1302 | 17 | 30 | USA | m | single | n | NaN | affection | 26-30 |
| 1380 | 18 | 30 | USA | m | single | n | NaN | enjoy_the_moment | 26-30 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 100517 | 13834 | 22 | USA | m | single | n | NaN | affection | 21-25 |
| 100520 | 13835 | 25 | USA | m | single | n | NaN | bonding | 21-25 |
| 100523 | 13836 | 31 | USA | m | single | y | NaN | achievement | 31-35 |
| 100526 | 13837 | 22 | USA | f | single | n | enjoy_the_moment | enjoy_the_moment | 21-25 |
| 100532 | 13839 | 24 | USA | f | single | y | NaN | affection | 21-25 |
4916 rows × 9 columns
single_series = single_df['predicted_category'].value_counts()
barplot = sns.barplot(x=single_series.index,y=single_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=90)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
married_df = age_gt_16_df[age_gt_16_df['marital']=='married']
married_series = married_df['predicted_category'].value_counts()
barplot = sns.barplot(x=married_series.index,y=married_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=90)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
sns.set_theme(style="whitegrid")
married_df['value'] = 1
married_df.head()
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/1074539863.py:2: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy married_df['value'] = 1
| wid | age | country | gender | marital | parenthood | ground_truth_category | predicted_category | range | value | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37 | USA | m | married | y | NaN | affection | 36-40 | 1 |
| 258 | 4 | 32 | USA | m | married | y | NaN | achievement | 31-35 | 1 |
| 354 | 5 | 29 | USA | m | married | y | leisure | leisure | 26-30 | 1 |
| 510 | 7 | 34 | USA | m | married | y | achievement | achievement | 31-35 | 1 |
| 678 | 9 | 61 | USA | f | married | y | NaN | affection | NaN | 1 |
category_grouped_df = married_df.groupby(['gender','predicted_category']).sum().reset_index()
g = sns.catplot(
x='predicted_category',
y='value',
hue='gender',
data=category_grouped_df,
kind='bar',
palette="dark", alpha=.6, height=6,
)
g.despine(left=True)
g.set_axis_labels("predicted_category", "count")
for ax in g.axes.flat:
for label in ax.get_xticklabels():
label.set_rotation(90)
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/283923920.py:1: FutureWarning: The default value of numeric_only in DataFrameGroupBy.sum is deprecated. In a future version, numeric_only will default to False. Either specify numeric_only or select only columns which should be valid for the function. category_grouped_df = married_df.groupby(['gender','predicted_category']).sum().reset_index()
married_df
| wid | age | country | gender | marital | parenthood | ground_truth_category | predicted_category | range | value | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37 | USA | m | married | y | NaN | affection | 36-40 | 1 |
| 258 | 4 | 32 | USA | m | married | y | NaN | achievement | 31-35 | 1 |
| 354 | 5 | 29 | USA | m | married | y | leisure | leisure | 26-30 | 1 |
| 510 | 7 | 34 | USA | m | married | y | achievement | achievement | 31-35 | 1 |
| 678 | 9 | 61 | USA | f | married | y | NaN | affection | NaN | 1 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 100493 | 13822 | 32 | USA | f | married | n | NaN | affection | 31-35 | 1 |
| 100502 | 13828 | 29 | USA | m | married | y | NaN | affection | 26-30 | 1 |
| 100511 | 13832 | 34 | USA | f | married | y | NaN | affection | 31-35 | 1 |
| 100514 | 13833 | 31 | USA | m | married | y | affection | affection | 31-35 | 1 |
| 100529 | 13838 | 38 | USA | f | married | y | NaN | enjoy_the_moment | 36-40 | 1 |
3613 rows × 10 columns
single_df['value'] = 1
single_df
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/3356902141.py:1: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy single_df['value'] = 1
| wid | age | country | gender | marital | parenthood | ground_truth_category | predicted_category | range | value | |
|---|---|---|---|---|---|---|---|---|---|---|
| 765 | 10 | 27 | USA | m | single | n | NaN | exercise | 26-30 | 1 |
| 936 | 12 | 25 | USA | f | single | n | NaN | affection | 21-25 | 1 |
| 1239 | 16 | 35 | USA | m | single | n | NaN | affection | 31-35 | 1 |
| 1302 | 17 | 30 | USA | m | single | n | NaN | affection | 26-30 | 1 |
| 1380 | 18 | 30 | USA | m | single | n | NaN | enjoy_the_moment | 26-30 | 1 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 100517 | 13834 | 22 | USA | m | single | n | NaN | affection | 21-25 | 1 |
| 100520 | 13835 | 25 | USA | m | single | n | NaN | bonding | 21-25 | 1 |
| 100523 | 13836 | 31 | USA | m | single | y | NaN | achievement | 31-35 | 1 |
| 100526 | 13837 | 22 | USA | f | single | n | enjoy_the_moment | enjoy_the_moment | 21-25 | 1 |
| 100532 | 13839 | 24 | USA | f | single | y | NaN | affection | 21-25 | 1 |
4916 rows × 10 columns
single_df_grp =single_df.groupby(['predicted_category'])['value'].sum()
single_df_grp
predicted_category achievement 1902 affection 1639 bonding 575 enjoy_the_moment 417 exercise 58 leisure 244 nature 81 Name: value, dtype: int64
married_df_grp = married_df.groupby(['predicted_category'])['value'].sum()
married_df_grp.values
array([1009, 2008, 199, 203, 31, 98, 65])
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
cleaned_hm_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/cleaned_hm.csv"
cleaned_hm_df = pd.read_csv(cleaned_hm_csv_file)
demographic_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv"
demographic_df = pd.read_csv(demographic_csv_file)
cleaned_hm_df.head()
| hmid | wid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 27673 | 2053 | 24h | I went on a successful date with someone I fel... | I went on a successful date with someone I fel... | True | 1 | NaN | affection |
| 1 | 27674 | 2 | 24h | I was happy when my son got 90% marks in his e... | I was happy when my son got 90% marks in his e... | True | 1 | NaN | affection |
| 2 | 27675 | 1936 | 24h | I went to the gym this morning and did yoga. | I went to the gym this morning and did yoga. | True | 1 | NaN | exercise |
| 3 | 27676 | 206 | 24h | We had a serious talk with some friends of our... | We had a serious talk with some friends of our... | True | 2 | bonding | bonding |
| 4 | 27677 | 6227 | 24h | I went with grandchildren to butterfly display... | I went with grandchildren to butterfly display... | True | 1 | NaN | affection |
cleaned_hm_df = cleaned_hm_df.dropna(subset=['ground_truth_category'])
merged_df = demographic_df.merge(cleaned_hm_df,on='wid')
merged_df.head()
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37.0 | USA | m | married | y | 28054 | 24h | My mother called out of the blue to tell me ho... | My mother called out of the blue to tell me ho... | True | 1 | affection | affection |
| 1 | 1 | 37.0 | USA | m | married | y | 57637 | 24h | I got to take the day off from my part time jo... | I got to take the day off from my part time jo... | True | 1 | affection | affection |
| 2 | 1 | 37.0 | USA | m | married | y | 64498 | 3m | my son had a great time on his 8th birthday. | my son had a great time on his 8th birthday. | True | 1 | affection | affection |
| 3 | 1 | 37.0 | USA | m | married | y | 80306 | 3m | I got to have a three day weekend which i spen... | I got to have a three day weekend which i spen... | True | 1 | affection | affection |
| 4 | 1 | 37.0 | USA | m | married | y | 88307 | 3m | My fiance proposed to me in front of my family... | My fiance proposed to me in front of my family... | True | 1 | affection | affection |
merged_df = merged_df[merged_df['country']=='USA']
merged_df['age'] = pd.to_numeric(merged_df['age'], errors='coerce')
merged_df['age'] = merged_df['age'].fillna(0).astype(int)
adult_df = merged_df[merged_df['age']>16]
married_df = adult_df[adult_df['marital'] == 'married']
married_df.head()
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37 | USA | m | married | y | 28054 | 24h | My mother called out of the blue to tell me ho... | My mother called out of the blue to tell me ho... | True | 1 | affection | affection |
| 1 | 1 | 37 | USA | m | married | y | 57637 | 24h | I got to take the day off from my part time jo... | I got to take the day off from my part time jo... | True | 1 | affection | affection |
| 2 | 1 | 37 | USA | m | married | y | 64498 | 3m | my son had a great time on his 8th birthday. | my son had a great time on his 8th birthday. | True | 1 | affection | affection |
| 3 | 1 | 37 | USA | m | married | y | 80306 | 3m | I got to have a three day weekend which i spen... | I got to have a three day weekend which i spen... | True | 1 | affection | affection |
| 4 | 1 | 37 | USA | m | married | y | 88307 | 3m | My fiance proposed to me in front of my family... | My fiance proposed to me in front of my family... | True | 1 | affection | affection |
married_df['value'] = 1
/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/430985916.py:1: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy married_df['value'] = 1
married_df.head()
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | value | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 37 | USA | m | married | y | 28054 | 24h | My mother called out of the blue to tell me ho... | My mother called out of the blue to tell me ho... | True | 1 | affection | affection | 1 |
| 1 | 1 | 37 | USA | m | married | y | 57637 | 24h | I got to take the day off from my part time jo... | I got to take the day off from my part time jo... | True | 1 | affection | affection | 1 |
| 2 | 1 | 37 | USA | m | married | y | 64498 | 3m | my son had a great time on his 8th birthday. | my son had a great time on his 8th birthday. | True | 1 | affection | affection | 1 |
| 3 | 1 | 37 | USA | m | married | y | 80306 | 3m | I got to have a three day weekend which i spen... | I got to have a three day weekend which i spen... | True | 1 | affection | affection | 1 |
| 4 | 1 | 37 | USA | m | married | y | 88307 | 3m | My fiance proposed to me in front of my family... | My fiance proposed to me in front of my family... | True | 1 | affection | affection | 1 |
sns.set_theme(style="whitegrid",palette="pastel")
married_series = married_df.groupby(['ground_truth_category'])['value'].sum()
barplot = sns.barplot(x=married_series.index,y=married_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=30)
for p in barplot.patches:
barplot.annotate(format(p.get_height(), '.1f'),
(p.get_x() + p.get_width() / 2., p.get_height()),
ha = 'center', va = 'center',
xytext = (0, 9),
textcoords = 'offset points')
married_df[married_df['ground_truth_category']=='affection'].to_csv("married_affection.csv")
single_df = adult_df[adult_df['marital'] == 'single']
single_df.head(2)
| wid | age | country | gender | marital | parenthood | hmid | reflection_period | original_hm | cleaned_hm | modified | num_sentence | ground_truth_category | predicted_category | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 125 | 10 | 27 | USA | m | single | n | 40557 | 24h | I went to see The Boss Baby with my little bro... | I went to see The Boss Baby with my little bro... | True | 2 | affection | affection |
| 126 | 10 | 27 | USA | m | single | n | 46740 | 24h | I played my favorite game with one of my favor... | I played my favorite game with one of my favor... | True | 2 | bonding | bonding |
# single_series = single_df.groupby(['ground_truth_category'])['value'].sum()
# barplot = sns.barplot(x=single_series.index,y=single_series.values)
# barplot.set_xticklabels(barplot.get_xticklabels(), rotation=30)
# for p in barplot.patches:
# barplot.annotate(format(p.get_height(), '.1f'),
# (p.get_x() + p.get_width() / 2., p.get_height()),
# ha = 'center', va = 'center',
# xytext = (0, 9),
# textcoords = 'offset points')
# single_df[single_df['ground_truth_category']=='achievement'].to_csv("single_achievement.csv")